Sunday, April 12, 2009

What you don’t see matters the most

Whereas it’s true that most JoomlaPack features are exposed by means of its graphical user interface, its the most important aspect is the backup core engine. What matters the most is having reliable backups every time, under any circumstances. This is what the backup core engine (CUBE) does and that’s why I had to improve it for the upcoming JoomlaPack 2.2.

Up until now, certain bottlenecks in the file packing code made it virtually impossible to use JoomlaPack on big sites running on slow servers. The reason was really simple. JoomlaPack had to scan all contained files and folders of a given folder in one go. If this operation took long enough (on slow servers it did), it would crash due to time outs. Same goes with packing big files. If many smaller files had been archived before hitting the big file, a timeout may had occurred.

Up until now we had to work around these limitations with certain fuzzy procedures, such as fiddling with Magic Numbers and changing to the Slow algorithm. The effect was that fewer operations would happen in a single step, reducing (but not always eliminating) the possibility of a time out occurring. Of course, this was a half-baked attempt to circumvent the problem, rather than solving it.

JoomlaPack 2.2.b1 will include my fixes to these problems. JoomlaPack is now smart enough to detect if a long file operation might cause a timeout, stop it, roll back and execute it in a new step, having the maximum amount of execution time available. There is also an improved filesystem scanner engine which detects large directories and forces their scanning to occur inside a single new step, to avoid timeouts.

The benefit of this approach is that JoomlaPack will now be able to work on sites where it wouldn’t work out-of-the-box and it will be substantially faster on sites where workarounds had to be implemented. For example, my own site required the Balanced settings in order to be possible to back it up. The backup took 15 minutes to complete. Now it works with the Optimistic settings and backs up in under 3 minutes! Wow!

Moreover, some servers couldn’t handle the amount of information required to be stored for large sites in the database between subsequent steps and crashed with a database error. I have included an option to force storage of temporary data in files instead of in database. This is required if you have directories deep down the fileystem hierarchy, containing more than 100-200 files.

Finally, there is a new feature you’re gonna love: Live Update. That’s right, JoomlaPack can detect if there is a new version released, notify you and install it with a single click. Please note that this feature requires that your server can draw data from other Internet sites. If RSS syndication with other sites works for you (i.e. you can display other sites’ RSS feeds in Joomla!) then Live Update will probably work as well. Also note that on servers where JoomlaPack’s “alternate installation” is required the single click update will fail. This is a limitation of the server.