Development Environment layout using Linux, Apache, PHP, and Subversion
Some of the age old questions I face lately are:
- What’s for dinner?
- Should I accept that friend request on Facebook for the friend of a friend of a friend that I knew 15 years ago?
- What’s the best development and test environment layouts for PHP using Apache as a web server with Subversion for version control for multiple developers?
Some of you may be asking yourselves the same questions. The choice of dinner is a personal one. I won’t go into that except to say that everyone loves a good burrito. Spicy! And you probably don’t care about my take on Facebook etiquette since your friends list probably dwarfs mine.
But I do have some definite thoughts on the layout of development environments. And I find that there’s a huge lack of information about this on the interweb, so here you go.
We use Linux, Apache, PHP, and subversion in our development environment and so these instructions will be biased towards these topics but I think you can apply this method using various other technologies.
I like to give each developer their own development web site and development database. I find it’s easier for everyone to have their own individual sandbox to play in. We give them each their own domain using their initials, something like rzdev.domain.com for me and vbdev.domain.com for another developer, Vinny Bag-o-Donuts. We set the Apache directories up on the Linux dev box in a similar fashion:
/var/www/rzdev.domain.com/
/var/www/vbdev.domain.com/
This has a few benefits. If I need to show Vinny something with my site development, I can just send him the link to http://rzdev.domain.com/broken-page. I can make changes to code, even major infrastructure code and not break anything for the other developers. We do the same thing with the databases, prefacing them with our initials.
Now, since our dev boxes use Linux, we set up Samba for sharing on these web directories. This means that all the devs can edit files and use source code management on the Linux server itself or on their Windows machines (we use either Eclipse or Zend Studio and create projects on the shares, that’s a whole different posting!).
This dev site layout is closely linked to the way we use Subversion for version control. When we start a new site or application, if we can split out the development evenly enough, we’ll just have everyone work from the trunk version of the code, with each developer working on their own little section. Each developer puts the trunk in their Apache dir and we edit the Apache configs to reflect this:
/var/www/rzdev.domain.com/trunk/
/var/www/vbdev.domain.com/trunk/
The root of the dev sites typically look like this:
/var/www/rzdev.domain.com/trunk/docs ( your Apache document root )
/var/www/rzdev.domain.com/trunk/lib ( non-public PHP library code )
When we commit code changes in Subversion, we have a hook that updates our main development site here:
/var/www/dev.domain.com/trunk
Again, that site can be seen on the web at http://dev.domain.com/ This way, we can do integration testing on our code to make sure our new code doesn’t break code from someone else within the dev site.
Now, the important thing here is that the Quality Assurance (QA) and Testing people ( if you’re lucky enough to have them ), don’t use any of these previously mentioned sites for their testing. Why not? Well, because if they’re doing a good job and are therefore sufficiently anal, they’re going to complain when code is changing on the site they’re looking at.
So we give them their own test site and database that’s viewed on the web at http://test.domain.com/ and setup in apache at:
/var/www/test.domain.com/trunk/docs
/var/www/test.domain.com/trunk/lib
The developers will meet and create the list of files and database changes that get moved over to the test site. How the actual moving is done doesn’t really matter. If you have the time and energy to set up some Ant or Phing tasks, that works great. But copying/rsyncing files and running some SQL on the test database works just as well. The most important part is that the developers meet to decide which part can go to test. Otherwise, you could have code going to test and eventually production that might not be fully vetted.
When QA finds bugs in test.domain.com, they can send them to the developers. The developers can instantly start working on fixing the bugs in their own dev space at rzdev.domain.com and not affect the other developers or the ongoing testing of the application. Pretty nice right?
Advantages of this approach
- Uses source code management
- Developers can unit test their own code
- Developers can do integration testing between each other’s code
- Developer A typically doesn’t destroy code or data that developer B is using
- Developers don’t destroy code or data that QA/testing is looking at
- Developers can both edit files and use source code management in either Linux or Windows environment
- Very scalable. Adding new developers into the mix is as simple as adding their respective sub domains and databases (of course, this can also be viewed as a disadvantage, see below)
- Less bugs make it to production
Disadvantages
- Lots of sysadmin overhead initially and with each subsequent domain added. You have to set up all those developer sites, rzdev, vbdev, etc. Same overhead when using branching within subversion. Plus, you have to setup all those databases and setup the config code to connect to the appropriate database for each developer domain.
- Lots of file space for all the sites and databases.
- Confusing for lone wolf and gunslinger developers who are used to overwriting production or each other’s development code (too bad for them!)
So what do you think? How do you setup YOUR PHP development environment?
..
Share this via del.icio.us, digg, email, etc.
Trackback for "Development Environment layout using Linux, Apache, PHP, and Subversion"
PHP articles
Google Reader
Del.icio.us bookmarks
15 Responses to “Development Environment layout using Linux, Apache, PHP, and Subversion”
By Tarjei on Dec 15, 2008 | Reply
Hi, IMHO the best setup is that each developer hosts his own dev environment on his local machine - preferably using the same OS as in production.
This will require either VmWare or linux on on the developers machine, but it does have some very important pros:
a) If something blows up, only one developer is affected.
b) The developer may be developing while offline or not in the office.
c) The developers learn a bit more about the challenges of the production environment (what - no 32 bit ffmpeg?) and make the changes needed at an earlier stage.
By Dado on Dec 15, 2008 | Reply
What he said.
Anyway, I’ve been working on two different setups (different firms). In first case, each dev has his own enviroment running locally, he’s responsible for setting it up etc and you can always point the local DNS zone to their machines. We all commit to SVN from which stuff gets deployed to devel machine on which tests are done. Then you have a staging machine on which the “clean” code should get tested again (this is like your ‘test.’ machine). In theory, no bugs on our producation machine.
Other setup: you have a Samba server which shares the sites on a local .web TLD, (for example, foo.web or bar.web), people mount the shares and work directly on them (simultaniously). When stuff is more or less done, then it gets committed to SVN (right from the share which is a working copy). This looks very prone to “Oopsie!” but in fact, seems to work for us. Pros are very simple editing and updating (for example, a dev says to the HTML guy “More foo, I need more foo here pronto!” and the HTML guy just does it, they both hit F5 to see is the foo plenty. Also friendly to designers which don’t like SVN, it seems.
Disadvantages should be obvious, but as I said, it seem to work.
By Stuart Herbert on Dec 15, 2008 | Reply
Interesting article.
I strongly prefer it when developers do their development locally. It gives them the most flexibility when working with new features, it makes it extremely easy for them to setup a separate environment per piece of work, and it forces them to learn both some basic sysadmin skills and an appreciation of basic sysadmin tasks. If the devs aren’t comfortable doing this, then imho they shouldn’t be writing code for publication on the Internet
Apart from emergency fixes, I don’t let our devs work in trunk. We work on the principle that trunk should always be as stable as possible, and should only contain code that’s either live or about to become live. Our devs have to integrate the changes from trunk back into their own branches daily, to ensure that their own changes don’t become incompatible.
This approach scales very well, minimises overheads (inc. sysadmins), and gives us great flexibility. An employee is taken ill suddenly? No problem, because his unstable work is in his own branch (where we can easily find it), not trunk (blocking up other code to release). We need to add more people to a project? No problem, because the code is in its own branch, and doesn’t have to be checked into trunk in order to be shared by the extra devs.
By Danny Verkade on Dec 15, 2008 | Reply
Our developers do our developing locally. But we do not have to create an virtual domain for every user and repository. the domain *.development.mbwp.nl maps to IP-adres 127.0.0.1, and Apache uses de vhost_alias mod, with the following line of code in the httpd.conf:
# include the server name in the filenames used to satisfy requests
VirtualDocumentRoot c:\development\%1
This maps project.development.mbwp.nl to c:\development\project
Although the approach mentioned above works fine, I’m wondering how you guys do the versioning of the database. How do changes in vbdev database get synchronized to the database rzdev uses?
By Jake on Dec 15, 2008 | Reply
I agree with Rich, this is the way we develop at my company and I love it. It ensures we do not create dependency on PHP modules/packages that will not reside on the production server. I believe your production and development servers should be identical (software). Allowing developers to develop locally can create anomalies.
Question:
Rich,
How do you prevent duplicate keys if your databases use autoincrement?
Database syncing between developers is where I’m struggling.
By Rich Zygler on Dec 15, 2008 | Reply
I’m surprised at the number of folks that allow development on local machines. What about backups of code that isn’t checked in? What about keeping track of PHP extensions and PEAR modules? What about caching solutions?
I’d much rather keep track of all that stuff in one place on the dev server rather then have to make sure everyone’s machine has the latest toolset running on it.
By Rich Zygler on Dec 15, 2008 | Reply
Jake,
Database synching really depends on the apps that you’re working with. We’ve never run into problems with it… but… Since we’re creating dumps of prod/test/dev data each night, a developer can if they want go and grab the latest database dump and import it into their own database.
By Dado on Dec 15, 2008 | Reply
@db sync
If you’re refering to DDL changes developers make, we’ve kept a file database.sql in SVN in which we’d add our changes (serial, once you’ve committed, you can’t alter that line, you must add a new line which changes the old one). Once you’re ready to commit, you append a schema version as the last line of the database.sql. Someone else then appends his changes and schema version line.
Also, we had a DB upgrade script written in python which would add a table with upgrade history to our DB and it would contain the complete upgrade history and the current “schema version”. When run, it would check if the schema in database.sql is newer than in the DB, if so, it would offer to upgrade. Pretty simple and also worked pretty well.
Some gotchas: the script would calculate the digest for each version to ensure you didn’t change the lines above your newly added changes.
Second, we had a separate database_lib.sql which contained stored procedures and checked it’s digest on every run, if it differed, it again offered to upgrade.
So, basically, you have a table
dbupgrade_log
- upgrade_datetime datetime not null
- version int not null
- schema_digest char(32) not null
- library_digest char(32)
database.sql:
alter table foo drop index foo_idx;
– #23
You should note that the “schema version” could be derived from the commit ID or even the line number in the database.sql. Also, if I’ve wrote it, I’d keep my digests inside the upgrade file, but I didn’t.
By Dominic on Dec 15, 2008 | Reply
I use git and several branches. If I’m working on GUI elements, I switch to the branch GUI. And so on. git has the advantage to checkout and switch the branch within the same folder. You can switch to an branch an your ‘dev’-build is up and runing on the server. Afterwards you can merge the branch with the ‘master’ and syncronise it with ‘git svn commit’.
I prefer to have a webdev-environment on my own machine.
To test (automatic unit tests) the commited code you can use http://www.phpundercontrol.org/about.html .
By Herman Radtke on Dec 16, 2008 | Reply
There are benefits to local development boxes. One example: I can test an upcoming release against a newer version of php while a colleague fixes a bug in a stable release that was built on an older version of php.
Keeping local machines up to date is not really a big deal for smaller companies.
By Martin Holzhauer on Dec 16, 2008 | Reply
I think your article hase very good points - i had a similar setup for my private projects.
i handeld the initial overhead disadvantage with shellscripting.
# ./createProject.sh newprojectname
which created a new svn repository, added a new domain to apache (thanks to vhost.d folder) commited the initial folderlayout and wished a good day
About Database versioning - A few month ago i found a very good blogpost[1] about handle a db schema with an version control system
The main points are, use a create.sql at the start and then add for every version of you schema an apply.sql and a remove.sql
i think to have an complete create.sql + the apply.sql and remove.sql for every version would be the best solution
about development - i worked with the folowing setups: own dev machine, dev on an sambashare and dev on a 1:1 copy in the same location as the live server.
the development via samba/any other share protocol has an great advantage over own development mashine - BACKUPs. really you always forget the backups on you own machine if you made smoething important
1: http://marksverbiage.blogspot.com/2008/07/versioning-your-schema.html
By kvz on Dec 17, 2008 | Reply
We distribute VirtualBox development images, which are clones of the production environment.
Allowing local development (on any desktop OS) for agility, and devs only submitting working code to SVN.
Of course it helps to decide who will work on what first.
More info in a writeup I did recently:
http://kevin.vanzonneveld.net/techblog/article/how_virtualization_will_improve_your_code/
By Ernie on Dec 20, 2008 | Reply
@Rich: I would hope developers are not keeping much uncommitted code on their machines. Keep code changes small, or for new work and larger changes, create a separate branch.
By Erik on Dec 23, 2008 | Reply
This article has been a god send… THANK YOU!! We’ve been researching for quite some time on how to set up subversion to work with multiple sub-domains and your article has really helped.
We have identical challenges in that due to certain server configurations, hardware settings, and db architecture, development on a local machine isn’t an option. No matter how much we try to emulate the server and it’s environment, we won’t ever have 100% certainty that we’ve covered everything. Hence the need for developing on the actual server.
Coincidentally our sub-domain set-up is nearly identical to yours but without the advantage of using subversion for the source control. I hate to admit this but we’ve been FTP’ing files back and forth between the various sub-domains for years… yeah I know… it’s ugly and dangerous. That’s why we’re implementing source control.
You mention using “Projects” in your posting… do you have any other write-ups that describe how you’ve set that up? Rather then re-invent the wheel we’re curious to know any tips or advice you might have in relation to this aspect of source control.
By Rich Zygler on Jan 8, 2009 | Reply
@Erik, Sorry for not getting back sooner — busy with holidays. For setting up projects, typically we have the dev site on the linux box sitting at /var/www/rz.project.com/trunk/. We have a Samba share at /var/www/. This enables me on a Windows box to map a drive, say P:\ to linux:/var/www/.
Mapping the drive is important because older versions of Zend Studio couldn’t create projects on shares without the mapping (this may still be the case). So when we create the new project in Zend Studio, we point it at P:\rz.project.com\trunk and voila! This lets us create a separate project for each domain in Zend Studio and it also lets us use multiple development servers, all mapped to different drives if we need.