Results tagged “MacOS X Server” from Bill's Words

Xserve is long dead.

Mac OS X Server is an app, not a standalone product anymore, and is a shadow of its former self.

So it would not surprise me if this announcement is the first step towards a partnership which could supply both iOS and Mac OS-friendly “big iron” server technologies for enterprise, an area which Apple clearly has no interest in pursuing.

I’m not impressed with Mavericks Server or Yosemite server, either, for that matter. First, Apple has moved all of the standard binaries and settings from their usual homes into the Server.app bundle and into /Library. While this certainly lines up better with the Apple “way,” it makes it a royal pain in the butt for those of us who have half a clue and know their way around a LAMP (or MAMP) system.

I decided to update our server from an Xserve to a mini about three months ago. The new mini arrived, and I thought, “This should be ‘Apple simple,’” and tried to migrate. It was a disaster, mitigated only by the fact that I had a backup of the original system—somewhere in the migration process, files on the original server get mangled. This shouldn’t be, of course, but…

I tried various methods of upgrading and none were successful. Open Directory gave me fits, and I spent hours trying all sorts of things, none of which were successful. So today, I decided to go nuclear: start from scratch on the new server. That way, nothing could go wrong.

Right?

Sigh…

Since Open Directory gave me many fits, and since I knew OD is somewhat finicky about DNS entries, I decided I’d start with the basics.

Setting the Hostname

What is the name of my host? Well, Apple, in their infinite wisdom, asked my DNS to tell what the host name is, instead of asking me. So the host name was wrong because it was based on an outdated entry in the upstream DNS. So the self-signed certificate had the wrong data, too. To fix this, I changed the upstream DNS name, deleted Server, deleted /Library/Server/, and rebooted. I then reinstalled Server.app and this time the server name and self-signed certificate data were correct.

As it turns out, I could have used Server.app to change the name and the self-signed certificate would have been regenerated. But I found that out too late.

Server.app Pro Tip: When changing hostnames, Server.app will generate a new self-signed certificate.

Setting Up DNS

Next step: DNS. This should be simple. I should just be able to import a zone file, but, alas, unless I migrated the server, nope. I thought about just typing in all the hosts and whatnot, but what a pain in the rear that would be. Also, you can’t use wildcards in hostnames. So instead, I turned on DNS, set the forwarders to 127.0.0.1 and the upstream servers, and looked at the files in /Library/Server/named.

Complaint: There’s no way to reorder the forwarding servers in Server.app, without retyping the whole list.

Complaint: You can’t type in an asterisk (wildcard) when editing a host name. So aliases like “*.eccles.net” can’t be used without manually editing the hosts file. Still true in Yosemite.

Complaint: Manually-entered wildcards get deleted from the hosts file if you edit the zone with Server.app.

Server.app Pro Tip: Editing zone files is possible, but any changes made in Server.app will overwrite most (if not all) edits.

Now for Open Directory

OK, now that my server knows who it is, it’s time to turn on Open Directory. A few clicks and that was done. I now had a fresh Open Directory master running. Now let’s import some users. (Since I never really monkeyed around with OD in any other version of Server, a plain vanilla OD Master is fine for me here.)

Server.app Pro Tip: Set the Directory Administrator user to diradmin and set the password to be the same as the server administrator password. If you’re like me, you’ll stand a much better chance of remembering these credentials that way.

Importing Users

I exported the users from my 10.6 server (select the users in Workgroup Manager, then use some other command which I can’t remember) and tried to import them using 10.9.

Server.app Pro Tip: Importing users is not found in the “Users” pane in the “gear.” (Why not?) Instead, it’s here:

in the Manage menu. Server.app kept griping about my username and password. My question is, What username and password? The dialog says “Admin Name” and “Password” but doesn’t give a clue which thing it is I’m trying to authenticate into. I assumed it was the server, and several times, I was wrong. I then decided it might be the OD server that I’m trying to authenticate into, and that turned out to be right.

Complaint: Server.app could use a better prompt than “Admin Name”. How about “Open Directory Server Administrator Name”? It’s long, yes, but it’s better to try to fit that into the window than frustrate the user, don’t you think?

Complaint: If that’s too long to fit, how about improving the error message? “Credentials could not be verified. Username or password is invalid” could just as easily say, “Open Directory server credentials…” to save me a few tries and Googling.

Server.app Pro Tip: This dialog:

is asking for the Open Directory administrator username and password (which you just created—see above).

While the import was a success, it left me with questions. First, I have several users with more than one shortname (most, in fact). What happened to these additional shortnames? And what do I do with the blank “E-mail address” box in each user’s information? Does something go there? Does something have to go there? What’s up with that? Let’s tackle each one of these separately.

About those multiple shortnames: It turns out that they are, indeed, imported into the new Open Directory server, but only the first (primary) shortname is displayed. I verified this by making test SMTP sessions and watching the SMTP logs. Messages to all of a user’s shortnames were successfully delivered. Yosemite note: not true anymore. See below.

Managing these shortnames is tricky, though, and can probably be accomplished with a command line tool of some sort, though I was unable to figure out how to do it. (I gave up after ten minutes of Googling.) I stumbled upon an Apple support page which describes how to edit Open Directory records with Directory Editor.

“Directory Edi… wha…? you’re saying, I’m sure. Yes, one of the older apps hidden away from most users is Directory Utility, which I never use other than to enable root user. So what’s changed to make it useful? It has a new pane called “Directory Editor” which allows Open Directory directories to be edited. (Clever name.)

You can find Directory Utility using the Apple-given instructions at the link above, or you can…

Server.app Pro Tip: Make an alias to /System/Library/CoreServices/Directory Utility and stick it in your dock.

In DU, you can edit everything about an OD entry (hence the reason it’s probably hidden from most users’ attention). Since the server is local (it’s on the same machine), authenticate into the node at “/LDAPv3/127.0.0.1”, as shown below:

Each user will have a RecordName which will correspond to the primary shortname. If you have any users with multiple shortnames, you’ll see that they have more than one RecordName. If you want to add another shortname, you can do so with the “+” button out to the right of RecordName, as shown below:

Server.app Pro Tip: Multiple user shortnames can be added, edited and deleted in Directory Utility. But this isn’t really useful in Yosemite.

How about that “Email Address” field in Server.app? What does it do?

I have no idea. [Though it turns out to be useful in Yosemite.]

When a user is created, it suggests the E-mail address based on the user’s shortname. If you change it to be different from the suggested address, it does end up being reflected in the OD entry, but PostFix (the mail server) has no idea what to do with it. E-mails addressed to the different address will bounce. E-mails addressed to shortname@domain.tld will be delivered.

Server.app Pro Tip: Leaving the E-mail address field in a user record blank is OK. Except in Yosemite, that is.

Yosemite update: I upgraded from Mavericks to Yosemite. It now ignores the multiple shortnames specified in Directory Utility (see above). For example, my primary E-mail address might be administrator@somedomain.net and I might have an alternate shortname, bill.eccles@somedomain.net specified in DU. In Mavericks, I had to add the bill.eccles shortname manually using DU, per the above. I could successfully receive E-mail to either address. The E-mail address field meant nothing.

However, when I updated to Yosemite, PostFix doesn’t have any idea about these other shortnames/addresses anymore, even though they do show in Directory Editor. Panic ensues when incoming mail to these alternate shortnames bounces. This problem is reasonably-easily fixed by adding them to the users in the User editor in Server.app. But if you’re confused because the “+” button is grayed out for the user you’re trying to edit, it’s because you’re not authenticated into the appropriate directory node.

At the top of the list of users, you’ll need to filter to show only the “Local Network Users.” Then you’ll be able to double-click and edit a user. The “+” sign will be enabled for adding more E-mail addresses to the user. This has the same effect as editing the “EmailAddress” for the user in Directory Utility and does not effect the “RecordName” list. It might be a good idea to go back and remove the extra shortnames in the “RecordName” list, but I don’t know. And I haven’t done that yet, either.

About Users’ Passwords

Passwords are lost in the export/import process. It seems that it should be possible to find the various hashes in the older version of the server using mkpassdb, but I can’t find enough corresponding entries to know that I’d make the new server totally happy. The next question is how to handle passwords, since my users use the server only for mail (via IMAP or POP) and won’t have OS X’s native password changing dialogs.

It turns out that there’s a reasonably easy way to handle that, too. If I turn on the default SSL website (in the Website pane, naturally), I have the option to let users change their passwords. I tested this path, and it works well. But because my users come from outside the local network and have to traverse my firewall (which means all port 80 or port 443 access can be directed to one machine only), I have to either (a) migrate all the web services from the old server to the new or (b) set up a special port for accessing this server for password changes. I’ve chosen this latter method in order to make accessing the password change page more difficult. There is no way to change the default server port number, so this change will be done entirely at the firewall, redirecting port N to port 443.

Moving Mail Services

Mail services are somewhat tricky, but now that you have your users moved over, you can pretty easily move the mail to follow them.

First, turn off mail services on both the new and old servers using the Server.app. Then, we have to move the data from machine to machine.

Mail data exists in two places. There’s the Postfix SMTP spool files (mail which is in the process of being delivered) and the Dovecot IMAP spool files (mail which has been delivered to the users’ mailboxes).

First, get the SMTP files from the old server:

sudo tar cf smtp.tar /var/spool/postfix

(ignore the warnings about tar format cannot archive this (type=0140000): Inappropriate file type or format These are sockets and won’t archive, nor would you want them to.)

Get the mailboxes:

sudo tar cf mail.tar /var/spool/imap/dovecot/mail/

Copy them to the new machine somehow, e.g.:

scp smtp.tar admin@192.168.1.2:~/smtp.tar
scp mail.tar admin@192.168.1.2:~/mail.tar

Now put them where they belong.

Most likely, you already have mail directories where they belong, but they need to be cleaned out to prepare for new data. So here we’ll delete the mail data directory (i.e., clean it out… permanently!) and repopulate it with the mail from the original server:

sudo rm -R /Library/Server/Mail/Data/mail
mkdir -p /Library/Server/Mail/Data/mail
cd /Library/Server/Mail/Data/mail
sudo tar xf ~/mail.tar --strip-components=5
cd ..
sudo chown -R _dovecot:mail mail

Then we’ll make the spool directory (if it isn’t there already) and populate it with the spool data from the original server:

sudo mkdir -p -m 755 /Library/Server/Mail/Data/spool
cd /Library/Server/Mail/Data/spool/
sudo tar xf ~/smtp.tar --strip-components=3

I think this all I did, but you may have to jigger your permissions and ownership so it looks like this:

home:spool admin$ ls -la
total 0
drwxr-xr-x  16 root      wheel      544 May 26  2014 .
drwxr-xr-x  13 root      wheel      442 Aug 10 12:17 ..
drwx------   2 _postfix  wheel       68 Jan  2 19:15 active
drwx------   2 _postfix  wheel       68 Dec  8 08:00 bounce
drwx------   2 _postfix  wheel       68 Feb 19  2010 corrupt
drwx------  18 _postfix  wheel      612 Aug 10 12:33 defer
drwx------  18 _postfix  wheel      612 Aug 10 12:33 deferred
drwx------   3 _postfix  wheel      102 Dec  2 13:55 flush
drwx------   2 _postfix  wheel       68 Feb 19  2010 hold
drwx------   2 _postfix  wheel       68 Jan  2 19:15 incoming
drwx-wx---   2 _postfix  _postdrop   68 Aug  8 02:01 maildrop
drwxr-xr-x  24 root      wheel      816 Aug 10 13:19 pid
drwx------  27 _postfix  wheel      918 Jan  1 14:01 private
drwx--x---   7 _postfix  _postdrop  238 Jan  1 14:01 public
drwx------   2 _postfix  wheel       68 Feb 19  2010 saved
drwx------   2 _postfix  wheel       68 Dec 31 21:59 trace
home:spool admin$

So what’s next? PHP, web services, and other things… but that will have to wait until a future article. This one’s already long enough.

It was frustrating.

I recently added OpenDNS to my list of tools, important mostly because I’ve now got boys who venture out into the wilds of the internet and are learning the ins and outs of adware, malware and images of a “questionable nature,” to put it mildly. (You can discuss the merits of this parenting technique amongst yourselves. This is the choice my wife and I have made.) I simply changed the DNS address being distributed with the DHCP leases to point at my MacOS X Server (10.6.x—the last of the great server versions…) where I forwarded non-authoritative queries upstream to OpenDNS. And all was good.

Between when I did that and a few weeks ago, suddenly Dealnews.com stopped loading. About halfway through the page load, no matter the device—iOS or Mac OS—it would hang. Images wouldn’t load. They wouldn’t load no matter if I were using Safari or Google Reader or Reeder or nothin’. And thus began my quest.

First, I used Safari’s Develop>Start Timeline Recording to watch what was and wasn’t getting loaded. Neat. And it showed me that I was not getting any content from s5.dlnws.com (or its brethren, s1-s4, either). Easy enough. That content would be getting blocked by OpenDNS, certainly.

Except that it wasn’t. In fact, if I pointed my Mac at OpenDNS directly (without the MacOS X Server BIND in between), the page loaded just fine, no problems, no hangups. Using the tools on OpenDNS, I checked to see that the dlnws.com domain wasn’t being blocked. It wasn’t. So clearly, whatever was in the way was happening in my Mac OS X Server’s BIND. But… what? It was time for some conf file fu.

The next step was to figure out how to log queries of my DNS server. In Mac OS X, it’s BIND, so editing things associated with named—the “name daemon”—is the route to take. I added this code into /etc/named.conf. (I can’t find where I found this code, but Google reveals several possibilities.)

logging {
        include "/etc/dns/loggingOptions.conf.apple";

/* added by WNE to allow query logging*/  
        channel query_logging {
                file "/var/log/query.log" versions 7 size 10m;
                severity debug 3;
                print-time yes;
                print-severity yes;
                print-category yes;
        };

        category queries {
                query_logging;
        };
/* end of stuff added by WNE */

};

And I restarted named with Server Admin. Nothing.

Googling told me to turn on query logging. Most instructions tell you to do something like rndc querylog which won’t even work unless you sudo rndc querylog which doesn’t work because the connection is refused on port 953. Hiiiiiii-yah! More Google-fu was required to discover that Mac OS X Server’s default installation of BIND listens on port 54. (See /etc/named.conf if you don’t believe me.) So sudo rndc -p 54 querylog solved that problem, and I began to watch the log in /var/log/query.log.

Sure enough, queries for s5.dlnws.com were showing up. But I still had no pageloads! Seriously, what’s going on here? I mean, when I try it on the iMac from my command line, I get the right answer, don’t I? I get:

the-imac:~ eccles$ dig @192.168.1.4 s5.dlnws.com
;; Truncated, retrying in TCP mode.
;; Connection to 192.168.1.4#53(192.168.1.4) for s5.dlnws.com failed: connection refused.

Ah… I don’t get the right answer! But I did get the right answer if I asked OpenDNS—and I got a huge list of responses. Apparently, s5.dlnws.com is attached to a content delivery network (CDN) with lots of IP addresses. Hmm.

Again, applying some Google-D-40 to the problem (Do you like what I did there? I just invented it.) and I found that A.P. Lawrence had a similar problem, never quite solved since it involved his router. Mine involved Mac OS X, but it still pointed me to the crux of the problem: most DNS queries work using UDP (unsigned datagram protocol), but the UDP packet cannot be longer than 512 bytes. Clearly, this response was much longer than 512 bytes and dig was trying to do it with TCP (transmission control protocol—fancier and more reliable, but with more overhead than UDP), but it was being rebuffed at the gates. BIND wanted nothing to do with answering a query on TCP.

I did all the Google-suggested things, such as netstat and lsof only to discover that BIND was supposedly listening on port 53, both protocols. It just wasn’t paying attention.

The solution to the problem lay in one commented-out line in /etc/named.confwhich I uncommented:

        query-source address * port 53;

If you uncomment this line and restart DNS (using Server Admin or whatever your preferred method is), the problem is solved. Though BIND was listening to TCP, it only wanted to accept queries on UDP.

Problem. Solved.

And now I can shop the bajeebers out of Black Friday. Whew!

This update is not PHP-neutral. Apple installs PHP 5.3.15 over your custom PHP installation.

I looked at the configure that Apple uses and it looks about like our usual configure info, save for a few options (see below).

So the steps required to make this all work with what we have from our previous expeditions, except this time I decided to try a 5.4.8 installation, which should keep us ahead of Apple for quite some time:

0. Keep copies of the Apple binaries and how they configured their installation:

   $ php -i > ~/php-config-2012.11.08
   $ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
   $ sudo cp /usr/bin/php /usr/bin/php.old
   

1. Get the 5.4.8 tarball and unpack it.

2. Move the libpng headers around a bit so that configure doesn’t find old headers for use with the new libraries:

   $ sudo mv /usr/X11R6/include/png.h /usr/X11R6/include/png.h_old
   

3. Append the new flags to the configure statement and configure away:

   $ ./configure  '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--sysconfdir=/private/etc' '--with-apxs2=/usr/sbin/apxs' '--enable-cli' '--with-config-file-path=/etc' '--with-libxml-dir=/usr' '--with-openssl=/usr' '--with-kerberos=/usr' '--with-zlib=/usr' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--with-curl=/usr' '--enable-exif' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/usr/local' '--with-png-dir=/usr/local' '--enable-gd-native-ttf' '--with-ldap=/usr' '--with-ldap-sasl=/usr' '--enable-mbstring' '--enable-mbregex' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--with-mysql-sock=/var/mysql/mysql.sock' '--with-iodbc=/usr' '--enable-shmop' '--with-snmp=/usr' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-xmlrpc' '--with-iconv-dir=/usr' '--with-xsl=/usr' '--with-pcre-regex' '--with-freetype-dir=/usr/X11R6' '--enable-zend-multibyte' '--enable-zip'
   

(PHP will gripe about ‘—enable-zend-multibyte’ and ‘—disable-dependency-tracking’, but I’ve left them in here so I can go back and investigate. Note that this may cause a problem—I just don’t know yet.)

4. Use four cores to make PHP. It’s a lot faster to use the -j 4 option, and if you have more cores, make the number match:

   $ make -j 4
   

5. Install.

   $ sudo make install
   

6. Restart Apache.

   $ sudo apachectl restart
   

7. Clean up what we did to the libpng headers:

   $ sudo mv /usr/X11R6/include/png.h_old /usr/X11R6/include/png.h
   

Done.

Alternatively, if you have already installed 5.4.8 before, all you need to do is:

0. Keep copies of the Apple binaries and how they configured their installation:

   $ php -i > ~/php-config-2012.11.08
   $ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
   $ sudo cp /usr/bin/php /usr/bin/php.old
   

1. Install.

   $ sudo make install
   

2. Restart Apache.

   $ sudo apachectl restart
   

Done!

This update is not PHP-neutral. Apple installs PHP 5.3.6 over your custom PHP installation.

I looked at the configure that Apple uses and it looks about like our usual configure info.

So the steps required to make this all work with what we have from our previous expeditions:

0. Keep copies of the Apple binaries and how they configured their installation:

   $ php -i > ~/php-config-2011.10.20
   $ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
   $ sudo cp /usr/bin/php /usr/bin/php.old
   

1. Get the 5.3.8 tarball and unpack it. (5.3.8 seems to be a good drop-in replacement for 5.3.6, which is where I was.)

2. Move the libpng headers around a bit so that configure doesn’t find old headers for use with the new libraries:

   $ sudo mv /usr/X11R6/include/png.h /usr/X11R6/include/png.h_old
   

3. Append the new flags to the configure statement and configure away:

   $ ./configure  '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--sysconfdir=/private/etc' '--with-apxs2=/usr/sbin/apxs' '--enable-cli' '--with-config-file-path=/etc' '--with-libxml-dir=/usr' '--with-openssl=/usr' '--with-kerberos=/usr' '--with-zlib=/usr' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--with-curl=/usr' '--enable-exif' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/usr/local' '--with-png-dir=/usr/local' '--enable-gd-native-ttf' '--with-ldap=/usr' '--with-ldap-sasl=/usr' '--enable-mbstring' '--enable-mbregex' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--with-mysql-sock=/var/mysql/mysql.sock' '--with-iodbc=/usr' '--enable-shmop' '--with-snmp=/usr' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-xmlrpc' '--with-iconv-dir=/usr' '--with-xsl=/usr' '--with-pcre-regex' '--with-freetype-dir=/usr/X11R6' '--enable-zend-multibyte' '--enable-zip'
   

4. Use four cores to make PHP. It’s a lot faster to use the -j 4 option, and if you have more cores, make the number match:

   $ make -j 4
   

5. Install.

   $ sudo make install
   

6. Restart Apache.

   $ sudo apachectl restart
   

7. Clean up what we did to the libpng headers:

   $ sudo mv /usr/X11R6/include/png.h_old /usr/X11R6/include/png.h
   

Done.

Alternatively, if you have already installed 5.3.8 before, all you need to do is:

0. Keep copies of the Apple binaries and how they configured their installation:

   $ php -i > ~/php-config-2011.10.20
   $ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
   $ sudo cp /usr/bin/php /usr/bin/php.old
   

1. Install.

   $ sudo make install
   

2. Restart Apache.

   $ sudo apachectl restart
   

Done!

(I wonder if I addressed 10.6.6? Guess not…)

As usual, as is clearly stated, this update is not PHP-neutral. Apple installs PHP 5.3.4 over your custom PHP installation.

I looked at the configure that Apple uses and it looks about like our usual configure info.

So the steps required to make this all work with what we have from our previous expeditions:

0. Keep copies of the Apple binaries and how they configured their installation:

   $ php -i > ~/php-config-2010.02.22
   $ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
   $ sudo cp /usr/bin/php /usr/bin/php.old
   

1. Get the 5.3.6 tarball and unpack it. (5.3.6 seems to be a good drop-in replacement for 5.3.3, which is where I was.)

2. Move the libpng headers around a bit so that configure doesn’t find old headers for use with the new libraries:

   $ sudo mv /usr/X11R6/include/png.h /usr/X11R6/include/png.h_old
   

3. Append the new flags to the configure statement and configure away:

   $ ./configure  '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--sysconfdir=/private/etc' '--with-apxs2=/usr/sbin/apxs' '--enable-cli' '--with-config-file-path=/etc' '--with-libxml-dir=/usr' '--with-openssl=/usr' '--with-kerberos=/usr' '--with-zlib=/usr' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--with-curl=/usr' '--enable-exif' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/usr/local' '--with-png-dir=/usr/local' '--enable-gd-native-ttf' '--with-ldap=/usr' '--with-ldap-sasl=/usr' '--enable-mbstring' '--enable-mbregex' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--with-mysql-sock=/var/mysql/mysql.sock' '--with-iodbc=/usr' '--enable-shmop' '--with-snmp=/usr' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-xmlrpc' '--with-iconv-dir=/usr' '--with-xsl=/usr' '--with-pcre-regex' '--with-freetype-dir=/usr/X11R6' '--enable-zend-multibyte' '--enable-zip'
   

4. Use four cores to make PHP. It’s a lot faster to use the -j 4 option, and if you have more cores, make the number match:

   $ make -j 4
   

5. Install.

   $ sudo make install
   

6. Restart Apache.

   $ sudo apachectl restart
   

7. Clean up what we did to the libpng headers:

   $ sudo mv /usr/X11R6/include/png.h_old /usr/X11R6/include/png.h
   

Done.

Alternatively, if you have already installed 5.3.6 before, all you need to do is:

0. Keep copies of the Apple binaries and how they configured their installation:

   $ php -i > ~/php-config-2010.03.22
   $ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
   $ sudo cp /usr/bin/php /usr/bin/php.old
   

1. Install.

   $ sudo make install
   

2. Restart Apache.

   $ sudo apachectl restart
   

Done!

Brent Simmons makes some good points in this article. Fortunately, the only time I ever got Fireballed, I was doing just about everything he mentions. The only problem with my being Fireballed is that I was (at the time) on a single DSL, so my Mac OS X Server on a dual G5 wasn’t the chokepoint. Nonetheless, the pages were served, peaking at 150 requests per second.

If only people who used WordPress would get a clue…

[Via DaringFireball.net.]

The developer preview of Mac OS X Lion (version 10.7’s big-cat name), to which I do not have access, includes the amazing installation option to install Mac OS X Server. What once was a $1000 (since dropped to $500) standalone product will most likely be included in the standard-issue Mac OS X package. You know, the one that costs about $99 to buy. Amazing.

While others (via DaringFireball.net) have doubted that this is really the case, I’m going to go on record as saying it’s not only true, but will also guess as to why.

Let’s gather some dots and then connect them, shall we?

First, Apple has all but eliminated its line of server products. Other than the dedicated Mac mini Server, there are no more server-specific hardware products to be had. The Mac Pro Server is essentially the same box as the regular Mac Pro, after all. XServe is dead. XServe RAID is dead, too. With no serious computing iron to offer, a server room is going to be devoid of Apple products. There’s food for thought…

Second, Apple is building a huge data center with lots of serious serving power for the iTunes store. They also know a lot about how to serve data between computers using MobileMe. To me, that adds up to a lot of learned-by-experience knowledge about something called cloud computing, where the data are not necessarily in one particular place but are all over the place. As long as you don’t see any difference between having the data locally vs. in “the cloud,” it shouldn’t make any difference.

Third, Apple has some intellectual property brewing around cloud computing. Googling “Apple patent cloud computing” yields a pretty satisfying list of things to look at. Apple has its head in the cloud, quite literally.

Fourth, rumor has it that Apple is going to introduce a plan targeted at small businesses which will supply faster turnarounds and loaner machines for a very-reasonable $500 per year. Neat.

Fifth, Apple will be supplying Mac OS X Server technologies with every one of its desktop machines. Every one of them. Not just a few high-end machines. All of ‘em.

Finally, the cursor blinks at the same rate that it used to and I still type slowly, even though the processor power available to me, the user, has grown immensely. We may have gigaflops and terabytes on our desktops, but we still vastly underutilize them in a typical business setting.

Now let’s connect the dots.

(Oooo! That sorta’ looks like a unicorn!)

I think Apple is aiming to eliminate the server room entirely. Furthermore, I believe that with Server on every desk (and eventually I think at least some of it will be a default part of installation, mostly hidden), Apple will move the server room out to the front office. What once was one or more pieces of dedicated server hardware and software will be distributed across the machines in the workgroup or business. This approach makes use of Apple’s cloud technologies and will utilize all of that unused—but already bought!—computing power that we have on each user’s desk.

When? It won’t happen instantly, or even soon. No, I’ll peg the release date for this massive shift in computing to the release of version 11 of Mac OS X—what’s that, two, three years? It’ll take a big shift in mindset of Apple’s customers to accept this kind of radical change in how we think of “servers.” Also, the technology has to catch up with the plan, such as the need to implement some sort of new underlying file system which is certainly a prerequisite for this kind of thing (ZFS, anyone?). All of this will require some positive track record, presumably which Apple’s starting right now with OS X Lion.

I don’t think I’m going out on a limb here. Think about the advantages to both the users and how I think Apple sees things, and it makes good business sense for Apple as well as a good user experience.

For one, there’s no need to buy super-powerful hardware to do server stuff, especially if you have spare gigabytes and gigaflops sitting around idle. Why buy redundant capacity? Not buying more computers saves the business some money, and narrows Apple’s product line significantly. That’s good for both us and for Apple.

By getting rid of the server room, we users save money and space and possibly the time or expense of a dedicated system administrator. Why not administer the whole thing yourself? Or perhaps just hire an Apple-certified Consultant on an as-needed basis and pay the minimal fee per year to get your business-essential hardware replaced/repaired quickly? That sounds remarkably inexpensive to me. That’s another good deal for Apple, its consultant network, and us users, too.

If you are going to buy into this cloud server thing, presumably you’re somewhat locked into the Apple ecosystem. That benefits Apple, certainly, but it could also be perceived as a benefit to the consumer like the tight integration of iPod/iPhone/iPad and iTunes has proven beneficial to the user experience.

And what if that big data center in North Carolina were to sprout a twin? Could that become a backup for your business data cloud? Short answer: yeah. Can you say subscription model?

As a final thought, I don’t believe that Apple has any interest in big business with this initiative. As we’re often told these days, the heartbeat of America is the country’s small businesses (sorry, Chevy) and that’s a huge market. Maybe this will be trickle-up technology, but I doubt it: the Microsoft juggernaut has that one wrapped up for the foreseeable future, and I think the sysadmin community (which certainly “knows this can’t possibly work”) will be extremely resistant to the decentralized server model, at least one that is this decentralized. Who knows? It may not work on the big business scale. But I think… well, never mind. You can guess what I think.

I’m sure that I’m missing a few benefits and a few points which suggest that Apple really is headed this direction.

And I might be completely and totally wrong.

But, Oh! How I don’t want to be wrong… It’s just too cool an idea for it not to be real.

(Sorry I’ve turned comments off for the time being. Stupid spammers thought I needed to see their crap on my blog, so until I get hooked up with Disqus, things will be quiet. Let me know via E-mail—contact info over there on the left under “Pages”—and I’ll post your comments as part of my original posts.)

Sorry it took so long to get around to this. By now, I imagine you’ve already discovered that, as is clearly stated, this update is not PHP-neutral. Apple installs PHP 5.3.3 over your custom PHP installation.

I looked at the configure that Apple uses and it looks about like our usual configure info.

So the steps required to make this all work with what we have from our previous expeditions:

0. Keep copies of the Apple binaries and how they configured their installation:

   $ php -i > ~/php-config-2010.11.30
   $ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
   $ sudo cp /usr/bin/php /usr/bin/php.old
   

1. Get the 5.3.3 tarball and unpack it.

2. Move the libpng headers around a bit so that configure doesn’t find old headers for use with the new libraries:

   $ sudo mv /usr/X11R6/include/png.h /usr/X11R6/include/png.h_old
   

3. Append the new flags to the configure statement and configure away:

   $ ./configure  '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--sysconfdir=/private/etc' '--with-apxs2=/usr/sbin/apxs' '--enable-cli' '--with-config-file-path=/etc' '--with-libxml-dir=/usr' '--with-openssl=/usr' '--with-kerberos=/usr' '--with-zlib=/usr' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--with-curl=/usr' '--enable-exif' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/usr/local' '--with-png-dir=/usr/local' '--enable-gd-native-ttf' '--with-ldap=/usr' '--with-ldap-sasl=/usr' '--enable-mbstring' '--enable-mbregex' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--with-mysql-sock=/var/mysql/mysql.sock' '--with-iodbc=/usr' '--enable-shmop' '--with-snmp=/usr' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-xmlrpc' '--with-iconv-dir=/usr' '--with-xsl=/usr' '--with-pcre-regex' '--with-freetype-dir=/usr/X11R6' '--enable-zend-multibyte' '--enable-zip'
   

4. Use four cores to make PHP. It’s a lot faster to use the -j 4 option, and if you have more cores, make the number match:

   $ make -j 4
   

5. Install.

   $ sudo make install
   

6. Restart Apache.

   $ sudo apachectl restart
   

7. Clean up what we did to the libpng headers:

   $ sudo mv /usr/X11R6/include/png.h_old /usr/X11R6/include/png.h
   

Done.

Alternatively, if you have already installed 5.3.3 before, all you need to do is:

0. Keep copies of the Apple binaries and how they configured their installation:

   $ php -i > ~/php-config-2010.11.30
   $ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
   $ sudo cp /usr/bin/php /usr/bin/php.old
   

1. Install.

   $ sudo make install
   

2. Restart Apache.

   $ sudo apachectl restart
   

Done!

As is clearly stated, this update is not PHP-neutral. Apple installs PHP 5.3.2 over your custom PHP installation.

I looked at the configure that Apple uses and there are two new flags, '--enable-zend-multibyte' and '--enable-zip'. If you just append these flags to the previous configure statement, all is still not well because there will be a mismatch between the libpng headers and libraries. Finally, Apple’s sticking 5.3.2 out there. Why not use 5.3.3, I say?

So the steps required to make this all work with what we have from our previous expeditions:

0. Keep copies of the Apple binaries and how they configured their installation:

   $ php -i > ~/php-config-2010.09.03
   $ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
   $ sudo cp /usr/bin/php /usr/bin/php.old
   

1. Get the 5.3.3 tarball and unpack it.

2. Move the libpng headers around a bit so that configure doesn’t find old headers for use with the new libraries:

   $ sudo mv /usr/X11R6/include/png.h /usr/X11R6/include/png.h_old
   

3. Append the new flags to the configure statement and configure away:

   $ ./configure  '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--sysconfdir=/private/etc' '--with-apxs2=/usr/sbin/apxs' '--enable-cli' '--with-config-file-path=/etc' '--with-libxml-dir=/usr' '--with-openssl=/usr' '--with-kerberos=/usr' '--with-zlib=/usr' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--with-curl=/usr' '--enable-exif' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/usr/local' '--with-png-dir=/usr/local' '--enable-gd-native-ttf' '--with-ldap=/usr' '--with-ldap-sasl=/usr' '--enable-mbstring' '--enable-mbregex' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--with-mysql-sock=/var/mysql/mysql.sock' '--with-iodbc=/usr' '--enable-shmop' '--with-snmp=/usr' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-xmlrpc' '--with-iconv-dir=/usr' '--with-xsl=/usr' '--with-pcre-regex' '--with-freetype-dir=/usr/X11R6' '--enable-zend-multibyte' '--enable-zip'
   

4. Use four cores to make PHP. It’s a lot faster to use the -j 4 option, and if you have more cores, make the number match:

   $ make -j 4
   

5. Install.

   $ sudo make install
   

6. Restart Apache.

   $ sudo apachectl restart
   

7. Clean up what we did to the libpng headers:

   $ sudo mv /usr/X11R6/include/png.h_old /usr/X11R6/include/png.h
   

Done.

This page is the manpage for sa-learn. However, I usually find myself reinventing the process I go through to train SpamAssassin every time I want to do it, so here’s a summary of what I do. You might find it useful; you might not. I, on the other hand, most certainly will.

First, I usually move mail messages that Mail.app misses into a mailbox somewhere. I also move the items it determines are Junk into that same mailbox.

Second, I archive that mailbox to a directory on my server, so there end up being one or more .mbox files in that directory. That directory is shared by the server, so it’s no trick to archive the mailboxes straight from Mail.app to the server.

Third, I login to the server and cd to the directory containing the mbox files and do the following:

$ sudo sa-learn --forget --mbox *
$ sudo sa-learn --spam --mbox *

I guess I use the former command to make sure that SA forgets whatever it knew (or I thought it knew) about the messages. The second teaches the filter.

That’s it! Now, next time I want to do this process again, all I will have to do is search my own blog and won’t have to reinvent the wheel.

There’s precious little on the Interwebs about sharing a fax modem using Mac OS X Server 10.6 (or 10.5 or 10.4…). Trust me, I’ve looked.

If you’re looking for the clues, here they are. I started out by monkeying around with /etc/cupsd.conf, got it to work, but ended up just doing this:

1. I created the fax printer using the usual MacOS X Server System Preferences. I also set it up to answer and receive, and that works OK. I then tested the fax printer to make sure it works. (The modem is a MultiModem USB modem, FYI, so I have no Apple Stick-of-Gum Modem problems to deal with. It is rumored that the Apple Stick-of-Gum modem won’t work with a 64-bit server.)

2. Use Safari to visit localhost:631/printers/. Selected “MultiModemUSB”.

3. Selected “Maintenance” and “Modify Printer” from the drop downs. Entered my administrator username and password.

4. After some Carousel spinning, clicked “Current Connection” and “Continue”.

5. Checked “Share this printer” and “Continue”.

6. Left all settings as-is and clicked “Modify Printer”.

7. Clicked the “Administration” tab.

8. Checked “Share printers connected to this system”.

9. Clicked “Change Settings”.

After cupsd restarted, I was able to browse on 10.6 client machines to see the fax machine using the Default browser!

Could it really be that easy? Is this going to work? Will Superman be able to save Lois Lane in time?

Tune in next time…

Mac OS X Server v10.6.3 Update installs Apple’s PHP v5.3.1 over your custom install.

If you used sudo make install before, just do it again. I can’t see that Apple did anything particularly unusual with their PHP build. Just don’t forget to sudo mv php.dSYM php in /usr/bin because of the screwy results of make install.

Your mileage may vary, of course, so proceed with caution.

This article turned out to be pretty helpful in understanding Open Directory, though, admittedly, I haven’t given Apple’s documentation a try just yet.

When Art Fisher of In Color had some problems with my instructions for installing PHP, he and I dove into finding the solution.

Here’s what we came up with, and instead of publishing a new article, I just amended the old one.

I just noticed on my 10.5 server that it installs PHP 5.2.11 over your custom installation for 10.5 boxes. See here for more info.

I had one minor problem when installing 5.2.12 on my 10.5.8 box, and that is because of a gripe caused by multiple conflicting values in some header files which are both pulled in for the build. If you have this problem, it’ll look like this:

In file included from /usr/include/arpa/nameser.h:59,
                 from /Users/admin/Installed/php-5.2.12/ext/standard/dns.c:62:
/usr/include/arpa/nameser8_compat.h:304: error: conflicting types for 'HEADER'
/usr/include/arpa/nameser_compat.h:99: error: previous declaration of 'HEADER' was  here
In file included from /usr/include/arpa/nameser.h:59,
                 from /Users/admin/Installed/php-5.2.12/ext/standard/dns.c:62:
/usr/include/arpa/nameser8_compat.h:304: error: conflicting types for 'HEADER'
/usr/include/arpa/nameser_compat.h:99: error: previous declaration of 'HEADER' was here
In file included from /usr/include/arpa/nameser.h:59,
                 from /Users/admin/Installed/php-5.2.12/ext/standard/dns.c:62:
/usr/include/arpa/nameser8_compat.h:304: error: conflicting types for 'HEADER'
/usr/include/arpa/nameser_compat.h:99: error: previous declaration of 'HEADER' was here
In file included from /usr/include/arpa/nameser.h:59,
                 from /Users/admin/Installed/php-5.2.12/ext/standard/dns.c:62:
/usr/include/arpa/nameser8_compat.h:304: error: conflicting types for 'HEADER'
/usr/include/arpa/nameser_compat.h:99: error: previous declaration of 'HEADER' was here
lipo: can't open input file: /var/tmp//ccqI3ov4.out (No such file or directory)
make: *** [ext/standard/dns.lo] Error 1

To rectify that problem, create a file called patchfile and paste the following text into it.

*** ext/standard/dns.c       2009-10-16 18:09:49.000000000 +0200
--- ext/standard/dns.c    2009-12-17 18:10:55.000000000 +0100
***************
*** 56,63 ****
--- 56,65 ----
  #undef T_UNSPEC
  #endif
  #if HAVE_ARPA_NAMESER_COMPAT_H
+ #ifndef HAVE_ARPA_NAMESER_H
  #include <arpa/nameser_compat.h>
  #endif
+ #endif
  #if HAVE_ARPA_NAMESER_H
  #include <arpa/nameser.h>
  #endif

(I got the code above from this bug filed on php.net.)

Then do patch <patchfile followed by the usual configure/make/make install cycle described in this post (for example). You should be back in business.

Instead of publishing a completely new entry here, I’m merely going to say that if you want to know how to add FreeType support to PHP, then go have a look at the old entry, which I’ve amended with instructions for adding FreeType.

Since I'm a bit behind on testing Mac OS X Server updates, here's the scoop, many moons late: Server Security Update 2010-001 breaks PHP custom installations.

Apple installs PHP v5.3.0 over your custom PHP installation.

Here's what I did to alleviate the pain associated with the upgrade:

Before I upgraded, I backed up my configuration information (which I can use to reconfigure PHP if necessary) and current PHP installation so I can downgrade in a hurry if I can't rebuild the new PHP quickly:

php -i > ~/php-config-2010.01.20
sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.old
sudo cp /usr/bin/php /usr/bin/php.old

I then did the update, which on two machines, one client and one server, left the machine hanging with a blue screen and the spinning gear. Forcing a restart seemed OK, and there were no log gripes to speak of.

If I were very concerned that PHP weren't working correctly and quickly, I'd have swapped the newly-installed Apple PHP with the old one and set it to running while I rebuilt PHP. I didn't need to do that, though, so I just proceeded with rebuilding and reinstalling 5.3.1.

Once the update was done, I looked at what Apple used to configure php...

php -i

...and noticed that it wasn't all that unusual. I then backed up the Apple stuff...

sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.apple
sudo cp /usr/bin/php /usr/bin/php.old

...and grabbed the 5.3.1 PHP tarball from PHP.net, php-ized GD, configured it, made it, and installed it per these instructions.

I then restarted Apache2 with Server Manager and PHP5 was happy again.

OK, folks, as you may recall, I got a new server capable of running Mac OS X Server 10.6. Problem is, it doesn’t know jack about my custom PHP or Perl installations on my 10.5 server, so I have to figure out what to do to setup these things again. Let’s go!

What did I do before?

First, what did I have installed last time? Well, let’s start with the applications I use. First, there’s Gallery. Then there’s Moveable Type, which is what you’re using right now to read this ‘blog. And I have a weather collection app that runs in PHP. Each of these has its own requirements:

Gallery requires a bit of stuff:

• a database—check! That’s MySQL.
• an image processing library—check? Previously, I used ImageMagick, but I note that Gallery 2 now supports GD, and, as we’ll see later, I’m using GD for other things. So perhaps there’s some (ahem) synergy to be had here. Keep GD at the top of the list, ImageMagick if required.
• mod_rewrite—should be a checkbox in Server Admin, so let’s hope it is.
• ffmpeg
• dcraw
• jhead
• infozip—I’ll skip this one.
• zip—probably check?

MT requires no extra PHP goodies, but has a pretty good list of Perl modules that are optional in addition to a few that are required. From the requirements page, we get this list:

Required Perl modules

• CGI
• Image::Size
• File::Spec (Version 0.8 or higher)
• CGI::Cookie One of the following Database Perl Modules is required:

…

• DBD::mysql (version 2.9005 or higher) - When using MySQL database

…

(This is what I’ll use, so I ignore the others.)

WARNING: DBD::mysql 2.9004 is not recommended nor supported by Six Apart due to the error: “Statement has no result columns to bind”. Later versions of DBD::mysql are recommended.

Optional Perl Modules

The following Perl modules support option functions. Use these modules for even greater functionality with Movable Type.

Archive::Tar Archive::Zip Crypt::DSA Crypt::SSLeay Digest::MD5 Digest::SHA1 File::Temp GD HTML::Entities HTML::Parser Image::Magick IO::Compress::Gzip IO::Socket::SSL - New in MT5 IO::Uncompress::Gunzip IPC::Run List::Util LWP::UserAgent Mail::Sendmail MIME::Base64 Net::LDAP - New in MT5 Safe Scalar::Util SOAP::Lite (Version 5.0 or higher) Storable Text::Balanced (Necessary for searches within a blog) XML::Atom XML::Parser XML::SAX

And my own weather application requires only JpGraph, but it needs

• GD
• FreeType 2.x and
• the fonts for FreeType to use.

The big, burning question in my mind is, What has Apple enabled in the default PHP installation? Used to be that GD was off by default, as was FreeType. Let’s issue a php -i to find out:

$ php -i

Holy crap. In the output on this 10.6.2 box, we find

gd

GD Support => enabled
GD Version => bundled (2.0.34 compatible)
GIF Read Support => enabled
GIF Create Support => enabled
JPEG Support => enabled
libJPEG Version => 6b 
PNG Support => enabled
libPNG Version => 1.2.37

Amazing! This was a long-standing gripe I had, and the result is that I don’t need to do anything fancy… just yet. But is everything else there?

In any case, first step: grab Xcode and install it. Then don’t forget to turn on CGI execution and the php5_module in Server Admin.

Check!

Now let’s do something simple and make JpGraph work. (A few hours of frustration pass… and I’m not kidding. It’s so simple, and yet sometimes, so hard.) It’s really rather simple.

JpGraph

First, grab the tarball for JpGraph. Extract it. In my other installation, I had decided (for some odd reason) to put the scripts in the root directory of the weather site, but upon reading the installation notes, I decided to locate them a little more sensibly. So I moved/renamed src to /Library/WebServer/CGI-Executables/jpgraph like this:

$ cd jpgraph-3.0.7
$ mv src /Library/WebServer/CGI-Executables/jpgraph

and changed the ownership and permissions so that it can be used by the webserver user (which is in the group _www):

$ cd /Library/WebServer/CGI-Executables/
$ sudo chown -R admin:_www jpgraph
$ sudo chmod -R 755 jpgraph
$ xattr -dr com.apple.quarantine jpgraph

Then I moved the Examples folder to the root Documents folder (because I have no sites up at this point):

$ cd jpgraph
$ mv Examples ../Documents/

And then had to make a symbolic link in the Examples folder so that the examples can find the jpgraph directory.

$ cd ../Documents/Examples
$ ln -s  ../../CGI-Executables/jpgraph/ jpgraph

I then pointed my webbrowser to http://localhost/Examples/testsuit.php and, Yes! I have graphs! There are some gripes about PHP with no support for TTF. And this is where things get interesting because what we have to do now is rebuild PHP from scratch, ensuring that we get TTF—as well as all of the other things that Apple includes, plus the stuff we need for other applications—built into it.

That’s a bit tricky, and we have to break down the configure line that Apple used in order to see what libraries we have to make.

PHP, Just Like Apple

Let’s try to build PHP just the way Apple builds it, with their configurations ‘n’ all. If you look at the output of php -i, you can see what Apple used to build the PHP installation distributed with Mac OS X Server 10.6.

Configure Command =>  '/var/tmp/apache_mod_php/apache_mod_php-53~1/php/configure'  '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--sysconfdir=/private/etc' '--with-apxs2=/usr/sbin/apxs' '--enable-cli' '--with-config-file-path=/etc' '--with-libxml-dir=/usr' '--with-openssl=/usr' '--with-kerberos=/usr' '--with-zlib=/usr' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--with-curl=/usr' '--enable-exif' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/BinaryCache/apache_mod_php/apache_mod_php-53~1/Root/usr/local' '--with-png-dir=/BinaryCache/apache_mod_php/apache_mod_php-53~1/Root/usr/local' '--enable-gd-native-ttf' '--with-ldap=/usr' '--with-ldap-sasl=/usr' '--enable-mbstring' '--enable-mbregex' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--with-mysql-sock=/var/mysql/mysql.sock' '--with-iodbc=/usr' '--enable-shmop' '--with-snmp=/usr' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-xmlrpc' '--with-iconv-dir=/usr' '--with-xsl=/usr' '--with-pcre-regex=/usr'

Before I forget—we won’t be doing the standard make/make install without first backing up the php binaries. I had problems making sure that the architectures of my Apache server and the PHP binaries matched, and if they don’t, Apache won’t load them. And then Apache won’t load and your webserver is borked pretty good until you get PHP working perfectly. With that in mind, onward!

The first thing to do is, of course, grab the PHP tarball from its source and unsmoosh it. I tend to keep all of this stuff in a new folder, Installed, in my home directory, by the way.

Ideally, we’d just do the configure command just like Apple. Given that it’s easy to download Xcode and PHP and try it all out on the laptop as I’m writing this, I decided to just dive right in and try the configure command as Apple did it and see what it gripes about. (Copy and paste the original Apple configure command to your command line, then edit out the /var/tmp/Apache… bit so that it’s just ./configure.)

The first thing it gripes about is not being able to find the PCRE headers file.

checking for PCRE headers location... configure: error: Could not find pcre.h in /usr

That’s because this option

'--with-pcre-regex=/usr'

is telling Configure to look for pcre.h where it just won’t find it. I Googled it and discovered that there are all kinds of fixes out there that involve copying a file that doesn’t seem to exist into somewhere else so that it can be found. Hmm. That doesn’t sound like the right approach. And Apple has a patch that doesn’t seem to work, either because it’s targeted at an older version of PHP (5.3.0) or because they place a libpcre somewhere that we don’t have it in a clean installation.

After a while of digging, I realized that some of the advice out there on the web is merely to copy the header file from php-5.3.1/ext/pcre to /usr/somewhere. Wha…? Why copy this file if it seems like the normal PHP installation has PCRE built in as an extension, i.e., no library needed, it’s self-contained in the installation? Then all we have to do is something like we do with GD, namely just say “Yes! I want PCRE!” and don’t worry about where it comes from.

(By the way, downloading and installing libpcre also works. But that’s overkill at this point, and it may introduce problems with PHP because the current PCRE is 8.0 and the one installed with PHP is 7.8, I think.)

So if you change the flag above to this:

'--with-pcre-regex'

the problem goes away and configure reports

checking which regex library to use... php

Whew. Now, on to other problems, because there are some, specifically:

configure: error: libjpeg.(a|so) not found.

These flags cause that particular problem:

'--with-jpeg-dir=/BinaryCache/apache_mod_php/apache_mod_php-53~1/Root/usr/local'
'--with-png-dir=/BinaryCache/apache_mod_php/apache_mod_php-53~1/Root/usr/local'

There’s a big problem: we don’t have the mythical /BinaryCache/apache_mod_php... directory, so we have to make our own jpeg lib and png lib. Ready for that? Thought so! Let’s go!

libjpeg

First, grab the libjpeg tarball from its source. Then extract it and throw it into your Installed directory or wherever you’re keeping all this stuff, and it’s important to keep it because as often as Apple wrecks custom PHP installations, you are going to have to do it all again someday.

Things have gotten a bit easier since I did this last time, about five years ago, and all you have to do is the standard configure/make/make install with one option, like this:

$ ./configure --enable-shared

8< snip!

$ make

8< snip!

$ sudo make install

All we have to do now is keep in mind that libjpeg is now located in /usr/local/lib according to make and can edit the flag appropriately.

'--with-jpeg-dir=/usr/local'

libpng

Again, grab the source for libpng from its source and unpack it. Let’s see if anything’s changed since I last did this with v1.2.8.

$ ./configure
-bash: ./configure: /bin/sh^M: bad interpreter: No such file or directory

Harumph. Well, according to the INSTALL file, we might want to copy the makefile that lives in scripts and use that. Problem is, after I tried it, I noticed that libpng wants to find libz at ../zlib, where it certainly isn’t. And according to my notes, that’s a problem I ran into before. Here’s how to fix all that:

$ cp scripts/makefile.darwin makefile
$ emacs makefile

(or pico or nano or vi or whatever…)

Change this

# Where the zlib library and include files are located                                                                                                            
#ZLIBLIB=/usr/local/lib                                                                                                                                           
#ZLIBINC=/usr/local/include                                                                                                                                       
ZLIBLIB=../zlib
ZLIBINC=../zlib

to this

# Where the zlib library and include files are located                                                                                                            
ZLIBLIB=/usr/lib
ZLIBINC=/usr/include
# ZLIBLIB=../zlib                                                                                                                                            
# ZLIBINC=../zlib

and save it. You have to do this because Apple has started to include (or has always included?) the zlib library, and it’s not in the /usr/local directory but is with the rest of the Apple-provided stuff in /usr. Then

$ make
$ sudo make install
$ ./pngtest pngtest.png

which should produce a bunch of lines with r’s and w’s. If it does, then, Yay! Continue onward.

The libraries end up in /usr/local/lib, just like libjpeg, so its options look similar:

'--with-png-dir=/usr/local'

And now let’s hope for the best.

Back to PHP

If you try again, things should work just fine with the exception of this error:

Notice: Following unknown configure options were used:

--disable-dependency-tracking

Check './configure --help' for available options

We can ignore that one, unless you’re particularly retentive and decide to eliminate it. Otherwise, the complete configure command looks like this:

./configure  '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--sysconfdir=/private/etc' '--with-apxs2=/usr/sbin/apxs' '--enable-cli' '--with-config-file-path=/etc' '--with-libxml-dir=/usr' '--with-openssl=/usr' '--with-kerberos=/usr' '--with-zlib=/usr' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--with-curl=/usr' '--enable-exif' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/usr/local' '--with-png-dir=/usr/local' '--enable-gd-native-ttf' '--with-ldap=/usr' '--with-ldap-sasl=/usr' '--enable-mbstring' '--enable-mbregex' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--with-mysql-sock=/var/mysql/mysql.sock' '--with-iodbc=/usr' '--enable-shmop' '--with-snmp=/usr' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-xmlrpc' '--with-iconv-dir=/usr' '--with-xsl=/usr' '--with-pcre-regex'

If configure runs OK, then try make. (If you’ve been trying make at various points along the way, don’t forget to make clean.)

It’ll fail. Don’t ask me how I know, I just know. My sources tell me you’ll see:

Undefined symbols:
  "_libiconv", referenced from:
      __php_iconv_strlen in iconv.o
      _php_iconv_string in iconv.o
      _php_iconv_string in iconv.o
      __php_iconv_strpos in iconv.o
      __php_iconv_appendl in iconv.o
      __php_iconv_appendl in iconv.o
      _zif_iconv_substr in iconv.o
      _zif_iconv_mime_encode in iconv.o
      _zif_iconv_mime_encode in iconv.o
      _zif_iconv_mime_encode in iconv.o
      _zif_iconv_mime_encode in iconv.o
      _zif_iconv_mime_encode in iconv.o
      _zif_iconv_mime_encode in iconv.o
      _php_iconv_stream_filter_append_bucket in iconv.o
      _php_iconv_stream_filter_append_bucket in iconv.o
ld: symbol(s) not found

Apparently, the only way to fix the problem that exists in PHP5.3.1 is to edit the file ext/iconv/iconv.c from this (which is down on line 185 or so—don’t mix it up with the first one in the file!)

#ifdef HAVE_LIBICONV
#define iconv libiconv
#endif

to this:

#ifdef HAVE_LIBICONV
#define iconv iconv
#endif

Now try make and see what happens. I think you’ll be pleasantly surprised.

The real litmus test will be to install it. Let’s first backup the Apple PHP and then install ours.

$ php -i > ~/php-config-2010.01.17
$ sudo cp /usr/libexec/apache2/libphp5.so /usr/libexec/apache2/libphp5.so.apple
$ sudo cp /usr/bin/php /usr/bin/php.apple

(When I did the php -i above, it griped with

PHP Warning:  Unknown: It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'America/New_York' for 'EST/-5.0/no DST' instead in Unknown on line 0

so I edited /etc/php.ini to read

[Date]
; Defines the default timezone used by the date functions                                                                                                         
; http://php.net/date.timezone                                                                                                                                    
date.timezone = America/New_York

All of the choices for date.timezone are at the URL shown in the .ini file.)

Now to Add FreeType…

…tomorrow. That’s enough for one day.

Updated! Here’s how to add FreeType (and fonts)

It turns out that adding FreeType support is very easy. First, make sure you install X11 support when you install Mac OS X Server. Then all we have to do is change our configure command with one additional switch. It ends up looking like this:

./configure  '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-dependency-tracking' '--sysconfdir=/private/etc' '--with-apxs2=/usr/sbin/apxs' '--enable-cli' '--with-config-file-path=/etc' '--with-libxml-dir=/usr' '--with-openssl=/usr' '--with-kerberos=/usr' '--with-zlib=/usr' '--enable-bcmath' '--with-bz2=/usr' '--enable-calendar' '--with-curl=/usr' '--enable-exif' '--enable-ftp' '--with-gd' '--with-jpeg-dir=/usr/local' '--with-png-dir=/usr/local' '--enable-gd-native-ttf' '--with-ldap=/usr' '--with-ldap-sasl=/usr' '--enable-mbstring' '--enable-mbregex' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--with-mysql-sock=/var/mysql/mysql.sock' '--with-iodbc=/usr' '--enable-shmop' '--with-snmp=/usr' '--enable-soap' '--enable-sockets' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--with-xmlrpc' '--with-iconv-dir=/usr' '--with-xsl=/usr' '--with-pcre-regex' '--with-freetype-dir=/usr/X11R6'

Sure enough, after a make clean/make/make install (and restarting Apache), you get TTF support enough for the JpGraph to acknowledge that it is installed, but it will gripe that Arial and Verdana aren’t installed. Which they aren’t. So it’s right.

So how do we go about installing these fonts? Simple! Since we’re using a Windows machine, all we have to do is copy them from Microsoft… oh… wait… that’s a problem, now, isn’t it?

Well, now… all we have to do (for real) is get the core web fonts, which include Arial and Verdana, from Microsoft or somewhere else and put them where Jp Graph can find them. There are two ways that I know of to do this.

1. Go to the Internet Archive and download each individual file as Microsoft used to distribute them, circa 2002. The page you’re looking for is here. Apparently, that’s how I found them about eight years ago, because I don’t have any knowledge of how to do:

2. Download the fonts from the Corefonts project. This technique involves downloading a “cab” file (which Microsoft seems OK with) and extracting the contents using some utilities. You might want to just copy the .ttf fonts from some conveniently-available PC.

JpGraph would like to see the fonts in /usr/share/fonts, so all we need to do is create that directory and stick the font files there. I have mine in a folder in my Installed directory, so here’s what I did.

$ sudo mkdir /usr/share/fonts         
$ sudo mkdir /usr/share/fonts/truetype
$ sudo cp * /usr/share/fonts/truetype/
$ ls /usr/share/fonts/truetype/

After that, if all goes well, you’ll be able to load the JpGraph examples and see all the fonts working just fine.

Really, that’s all there is to it…

Updated! Compiling PHP with libpng 1.4.x

Art Fisher of In Color commented below that he had problems with this error during the make when he tried my instructions:

Undefined symbols:
  "_png_check_sig", referenced from:
      _php_gd_gdImageCreateFromPngCtx in gd_png.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [libs/libphp5.bundle] Error 1

I had run into that problem, too, and filed a bug against PHP 5.3.1. I guess the guys at PHP.net consider a bug that they found and fixed in a not-so-public version of PHP (a nightly build of about a month ago) “bogus,” so they marked my bug as “bogus.” I’d have preferred “closed,” but semantics aside, the problem above is caused by the removal of the long-deprecated _png_check_sig from libpng as described in the release notes, here.

Art and I worked together to figure out what was going on and discovered that if you edit line 148 of ext/gd/libgd/gd_png.c from

       if (!png_check_sig (sig, 8)) { /* bad signature */
                return NULL;
        }

to

       if (!png_sig_cmp (sig, 0, 8) == 0) { /* bad signature */
                return NULL;
        }

PHP will compile fine.

He then found some other problems, one of which you’re most likely encountering and for which I’ll share the solution below.

Updated! PHP not installing a CLI binary

After you’ve done all of that stuff above, you’ve probably fired up your web browser and run a simple phpinfo() script and discovered that you did indeed succeed.

To show off your command-line prowess to you spouse (who will be suitably impressed when you say, “Look, Dear! I successfully compiled and installed PHP version 5.3.1!”), you do a php -i which shows that you didn’t succeed at all! Instead, php -i reports that the old version, compiled by Apple, remains! I looked, and sure enough, the old version of php remained untouched, but there was a strange, and very functional, php.dSYM file sitting right next to it in /usr/bin.

Huh?

After Googling a bit, I discovered this web page which describes an esoteric problem that the libtool has in Leopard, and now in Snow Leopard, where gcc can’t seem to fully optimize PHP without mis-generating a .dSYM file. Or something like that. That guy on that page describes a way to not-quite-fully-optimize PHP and avoid the problem. Others have described a simpler way to solve the problem which yields a fully-optimized PHP. All you have to do is symlink php to the php.dSYM file.

In case you haven’t moved the old php out of the way yet (in which case that php -i above wouldn’t have worked in the first place), do this

$ sudo mv /usr/bin/php /usr/bin/php.apple

then this to make the symlink:

$ sudo ln -s /usr/bin/php.dSYM /usr/bin/php

That way, if you reinstall PHP again (as Apple has a tendency to step on our custom PHP installs, you’re certain to do so…) the newly-created php.dSYM will still link to php commands. Furthermore, if Apple does decide to fix the libtool/gcc problem, then php will be replaced and the problem goes away anyway.

Another Error: Warning! dlname not found in /usr/libexec/apache2/libphp5.so.

If you see this

Warning!  dlname not found in /usr/libexec/apache2/libphp5.so.

in the middle of the make install step, and yet PHP seems to work fine on both the command line and via Apache, as best Art and I can figure out, it’s OK. Don’t worry.

Updated! Libpng Header Version Problems

If you did what I did and installed libpng 1.4.x, then you have had this problem and probably don’t realize it. When you compile PHP and point it at /usr/X11R6/inclde to pick up the FreeType headers, it also picks up an old, 1.2.x, version of the libpng headers, but still builds PHP with the new version of the library itself, which is in /usr/include. And that’s a problem.

I’ve not been able to figure out how configure decides where to find what, nor how it could even be directed to do otherwise, but my workaround was to do

$ sudo mv /usr/X11R6/include/png.h /usr/X11R6/include/png.h_old

on a temporary—or maybe permanent—basis. I’m not sure I’ll ever need to do anything “real” with X11, so I think it’s OK to leave it that way. But, if you’re, er, retentive about things, you’ll want to put it back when you’re done making a new PHP.

By the way, I will be creating a new entry real soon now that summarizes all of these steps because as I’m doing it fore real, I realized it’s a real pain to read through the narrative to get to the heart of the process.

Well, folks, my dad decided to really splurge and bought our family a new-to-us Intel Xserve. A “Late 2006” model with four cores of Xeon power, it does hum along. Speaking of hum, this puppy sounds a lot like a jet plane when it’s running full bore. Fortunately, it lives in the well-insulated computer room where finding it just the right spot has been somewhat difficult. It is a big box, after all, about the thickness and width of a pizza box, but about twice as deep. Try finding counter space for two large pizzas and you’ll have some idea of the challenge. I’m a creative kinda’ guy, though, so I’ve found a solution that works nicely and will actually reduce the apparent space taken in the room by the various computers.

So far, the only problem I’ve had is trying to find disks for it. The Apple Drive Modules for this box went out of production a long time ago, and woe to the sysadmin who needs a new one! No worries, though, as the box has two FIreWire 800 ports and two USB 2.0 ports. And if I ever want “faster,” I’ll put in a eSATA RAID controller. But in the meantime, the USB 2.0 will do. (Strangely, most of my drives’ FireWire 400 controllers are toast, causing bus resets at an alarming rate. I’ll stick with USB 2.0, which is plenty fast for serving the files around the house.)

What does this mean for you? Well, the best part of the deal was really the price, since we got this box for $1700 with a copy of Mac OS X Server 10.6. w00t! I’ll be able to continue updating you on whether Apple destroys your custom PHP installs… as soon as I get my own custom PHP install put in.

That’s this morning’s project, and I’ll let you know how it goes.