Getting closer to 1.2.b1

After having done a lot of work the past few days, we are getting closer to releasing 1.2.b1. The most notable additions are:

• Support for accessing the backend with HTTPS (still needs some intensive testing)
• Single file exclusion filter
• Backing up without using AJAX

The codebase had the HTTP protocol hard-coded in the URL generation part, which made using the component impossible for people accessing their backend through HTTPS. It only got worse if you were actually enforcing the use of HTTPS with an appropriate .htaccess file. The fix I did was rather rudimentary: JoomlaPack chooses its protocol prefix based on whether the server has informed PHP that we were called through the SSL protocol. This feature still needs extensive testingto make sure there are no loopholes in the code and that it supports non-standard HTTPS ports (which means, anything but port 443).

As far as the single file exclusion is concerned, this was an asked for feature for quite a while. Many people want to exclude just that pesky unreadable file their host puts on the website's root. Or, maybe, those two big video files from the downloads directory. You get the picture... The interface is slightly weird, mainly because it just doesn't look like Joomla!. There are two panes, the left one displaying directories and the right one displaying files. You click on the directories to visit them, click on the files' checkboxes to toggle the exclusion status. Pretty easy, but I'd certainly prefer to code a proper tree view on the left side, a la Konqueror, or Windows Explorer :/

The "backup without AJAX" was a last minute addition, after some user had terrible problems with using AJAX during the backup, for no apparent reason. This can happen due to a million reasons, one of them being that the server is overpopulated. Instead of giving up and let frustration get the better of you, you can just switch to the brand new "JavaScript Redirects" backup mode. This is very simple, indeed. After each piece of work performed (a "step" in JoomlaPack jargon), a JavaScript redirect is issued to make the process proceed. It's not the same as the front end backup because it neither uses HTTP 301 headers, nor does it output a blank page during the backup. On the contrary, it outputs the familiar "backup status" messages, albeit in a funny looking way, yet.

All these exciting features were added in SVN 113, just a few minutes ago :)

Introducing Kickstart and JPA

One of the most notable additions in JoomlaPack 1.2 - which is still in Alpha - is that we now offer a renowned archive format, the JoomlaPack Archive (JPA) format. Its major benefit is that it can be more reliably created in the context of PHP script, such as JoomlaPack. On the downside, it is a custom format, without support from external tools. In here I will tell you not only how you can extract the archive, but how you can easily restore a site's backup too! But first, let's see how and why we got there...

Most people upload files to their sites using FTP. Despite of being the most widely used file transfer protocol for uploading files, it comes with some drawbacks. The most prominent is slow uploads when you have lots of small files. A Joomla! installation is a typical example, with over 3000 files occupying a mere 15Mb. No matter how much bandwidth you have, uploading takes at best half an hour.

Wait a minute! Half an hour for 15Mb worth of files?! It reminds me the dark days when I used PSTN to connect to the internet... But, what else can you do? If you upload the backup archive itself, it certainly gets uploaded fast (the more bandwidth you have, the faster it's uploaded), but you can't "run" the ZIP/JPA file. Or can you?

Using this thought, I decided to create a script which can "bootstrap" the installation process. I won't take credit for the original idea; I took it from the way software such as Joomla.Start works. I just enhanced the original idea in order to:

• use AJAX-powered, multiple step unpacking, so that it can operate on huge archives
• support both ZIP and JPA formats
• handle .htaccess renaming automatically
• delete itself, the installation folder and the archive after the installation is complete

It has one fundamental limitation, which is actually a limitation of PHP itself: the Safe Mode has to be turned off, or the folder on which you'll extract the archive has to be owned by the same user as the one the web server runs under. Just setting the permissions won't help it.

So far this software is alpha as well and you can get it through our SVN repository (directory kickstart). In order to use it just upload the backup archive and the kickstart.php to your server, then visit http://www.mysite.com/kickstart.php. Just follow the instructions from that point. In the end you'll have a fully functional site set up in -literally- minutes!

The roadmap to JoomlaPack 1.2

Since mid-March there has been an ongoing development effort towards version 1.2 of the JoomlaPack backup component. There are several new features and enhancements in this version. This post is a brief summary of what we intend to implement. I thought this is the most appropriate very first post on our brand new blog :)

Major features for 1.2

1. Native mode for both Joomla! 1.5.x and Joomla! 1.0.x using the same codebase
2. Database Table Exclusion
3. JoomlaPack Archive Format
4. "Kickstart" script
5. Reworked database only backup
6. Single File Exclusion
7. Front-end database only backup
8. Multiple Database Backup

The first five features have already been implemented in SVN revision 100 (most of them have also been available in Alpha 2 released earlier in May), the next two are scheduled for Alpha 3 and the complete feature set will be available on the Beta releases.

Explaining the features

Native mode. So far JoomlaPack has been a component written for the 1.0.x branch of Joomla! and would only work on J! 1.5.x with the Legacy Plugin enabled. Acknowledging that Legacy mode is a resource hog, we decided to restructure the component in order to work as a native J! 1.5 component too. We did make it and now there are two packages disseminated for each release. The one ending in -j10 is a Joomla! 1.0.x native version, whereas the one ending in -j15 is the brand new Joomla! 1.5.x native version.

Database table exclusion. Some webmasters are obliged to run several J!-powered websites on the same database, or they share J!'s database with other scripts' tables. A long-running wish on their part is the ability to exclude arbitrary tables from the backup set. Your wish is our command!

JoomlaPack Archive Format. You have certainly noticed that CRC calculation gives certain hosts a really hard time: invalid CRC calculations cause decompression to fail, high server load on CRC calculation causes JoomlaPack to crash and stuff like that. This is why I thought of simplifying the archive format to something similar to a trimmed-down ZIP file, which doesn't contain the Central Directory record (saving some space and reducing CzipCreator's potential failure points) and having no CRC stored. There is an option to toggle between ZIP and JPA and there is also going to be a PHP unpacking script for JPA archives. Right now, the compression part is ready, the extraction part is under development.

Kickstart script. Currently, as soon as the user downloads the backup archive he has to extract it, upload it to the new server through FTP, fix permissions, rename .htaccess, run the installation, remove the installation directory, restore .htaccess and access the restored site. This is tedious. How about transferring the ZIP to the target server, running a script locally which extracts the archive, launches the installation and removes the installation script afterwards? This is the concept behind what I call "Kickstart". Think of it as a bootstrapper for the installation process. Kickstart is available in the SVN, requires PHP Safe Mode to be disabled and currently lacks documentation.

Reworked database only backup. Most people intended to use this feature to grab a phpMyAdmin-usable database dump of their site. The standard JoomlaPack functionality was to create a database dump suitable for use with JoomlaPack Installer (JPI & JPI2) or the standard Joomla! installer. This behaviour has changed to what most of the users expected: a database dump usable by third party MySQL utilities.

Single File Exclusion. On rare occasions we face hosts storing account configuration files on the server's root. To make things worse, these are owned by another (system administrator) user account, making them unreadable, occasionally causing JoomlaPack to fail. Some other times there is just one pesky big file we'd like to exclude from, let's say, DocMan's documents folder.

Front-end database only backup. Self explanatory, it is. Right now you are only to backup your entire site using front-end only tools. Now you're going to be able to backup just your database.

Multiple Database Backup. I have a site which hosts Joomla! alongside a home grown script. This script uses a different database but I need it for my J! site to function, as its functionality is embedded with com_wrapper. Currently, there is no way to backup another database. This is not very hard to do and at least one user asked for it, so I thought "let's do it". And it's a feature no other backup component has!