Tools of the trade

A topic I was willing to cover for quite a while is the toolchain used to build JoomlaPack. Some programmers have even asked me what are the best tools to build Joomla! extensions. While probably not perfect, the combination I'll present here has worked wonders for me.

• Integrated Development Environment. After having tried a number of different PHP IDE's, I settled for Eclipse PHP Development Tools. An Open-Source project by Zend, the makers of the PHP core engine, it uses the famous Eclipse IDE to provide a thorough PHP IDE. Another alternative is PHPeclipse, but lacks some features which facilitate development and management of large and/or complex problems.
  
• Version Control. Tracking your changes and being able to roll back when pooh happens is a must-have for anything more complex than "hello world" software. I regularly use Subversion (affectionately called SVN be everyone). In order to integrate it with Eclipse, I use Subclipse. By the way, this makes up for the best Subversion client I've used so far!
• Testing server. Every developer needs a local testing server. The easiest solution is to use a prepackaged, easy to install solution, like XAMPP or WAMPserver.
• Reference book. Joomla! 1.5 offers a very rich and powerful framework. Before setting out to develop an extension for it, it is very useful to read a book on Joomla! extension development. The two books I have read so far are Mastering Joomla! 1.5 Extension and Framework Development (wonderful reference book) and Learning Joomla! 1.5 Extension Development (more like a tutorial, recommended for beginners). Of course you can shop for something else and it probably takes a few tries to find the one which is best for you.
• Debugging tools. I regularly use J! Dump to gain insight on my code. To be perfectly honest, once I found out about it and began using it, I started to wonder how could I code without it! It replaces your needs for ugly calls to var_dump(), die(), etc.

That's about it, guys and gals. Developing for Joomla! is neither hard, nor expensive. All the tools, save the books, are Free and Open Source software.

Have fun writing some high quality code!

Taking site backup... to the desktop!

Lately I have sporadically "leaked out" some of my thoughts on JoomlaPack's future and features I am working on lately. One particular feature, is providing XMLRPC services. Why is it so important, you might wonder? Well, let's see what XMLRPC is all about.

Most people have published on a blog using a desktop application, be it ScribeFire, Windows Live Writer, w.bloggar, etc. The common feature of all these tools is that you are writing your article off-line and when you're ready you can publish it on your blog. The magic behind the scenes is XMLRPC. This buzzword really means that the client (your blogging software) sends a chunk of XML data to the server (your blog). This is specially formatted so that the server understands it has to perform an action (in this case, post an article). When it's done, it sends a response in XML format back to the client. The client processes it and tells you that your article is posted.

It wouldn't be so much of a deal if it weren't for the XML format itself. It is generic enough to cater for every use imagineable, but it's also very robust that it can be very well described. It is analogous to paper in written communication. While it does not provide the language, it provides the medium on which the language can be carried between the two parties involved. The choice of language (and the interpretation of the messages) is left to the two parties. But it's really cool to know that the message will always be written on paper, neither on stone, nor on papyrous.

Joomla! 1.5.x has two built-in XMLRPC plugins for posting articles. The first is the Joomla! XMLRPC services plugin and the other one is the Blogger API XMLRPC services. This way you can use an off-line blogging tool with Joomla!. In fact, it's the best way to have a newbie post articles in a Joomla! site, because it carries virtually no risk of messing up settings.

This is all very cool, but you are probably wondering where does JoomlaPack fit in? What does this article posting thing got to do with site backups? Well, remember when I said XMLRPC is very generic? It is so generic it can be used any time you want to tell the server to do something and inform you on the outcome. For example, we could tell the server to load JoomlaPack's engine (CUBE), starting a backup and returning us backup status information. In fact, this allows us to control the backup process from any networked device as long as it can connect to our site's server.

The missing piece of the solution is the client part. Ideally, a client would run on any operating system (cross-platform) and be able to backup any site on which JoomlaPack runs. It should even grab a local copy of the backup archive for safekeeping. This piece of software would truly take site backup to the desktop!

Today, in SVN revision 313 (notably, this revision number is the same as Donald Duck's car registration number - I'm a lifetime fan of Donald :D - and has a number of notable properties ), I have commited the first part of the solution, the XMLRPC services plugin. I will now start writing the desktop application in Lazarus, a cross-platform programming environment similar to Delphi. My expectation is that this will be stable somewhere in mid-Q1 2009.

Stay tuned. JoomlaPack is about to redefine site backup.

A new tool in the 'hood!

You talk, we hear. We hear you and we think hard. Then, we come up with a solution, code it, test it and release it. That sums up the development process here at JoomlaPack Developers.

This time, we heard a lot of you complaining that sometimes backup archives are hard to extract. It seems that PHP4 users have a lot of grief with ZIP files, because on their platform the larger files appear to be corrupt with most traditional archivers. A lot of people who don't have an option but to use JPA files complain - rightfully - that this format seems arbitrary and lacks support from visual tools. You talked. We heard you.

Please, welcome JoomlaPack eXtract! eXtract is the latest addition to our utilities collection, with a twist. It is visual extraction wizard running on the Windows(tm) platform, similar in look and feel to the integrated Windows' ZIP extraction wizard. eXtract can extract ZIP and JPA backup archives generated by JoomlaPack. It comes with an installer to make your life easier during installation. Despite being version 0.1, it's usable.

Download eXtract now from our JoomlaCode.org FRS or the SourceForge.net mirror.

Updating Joomla! websites with JoomlaPack Kickstart: a step by step guide

This article is long owed to the community. I apologise for taking forever to write it down, but development of the 1.3 version was much more higher in my priority list. In the light of the new Joomla! 1.5.7 release - and having two sites demanding urgent update - I have decided to share the secret with you: It's upgrade time and Kickstart can be used to apply the Joomla! update patches to your site easily. It's really simple, too. Let's go!

First of all, you'll need the SVN version of Kickstart which supports plain file or FTP file access modes, integrates AJAX and JavaScript redirects modes and includes a few more improvements. Since this is not released yet, you can download a developer's preview - but very stable already! - version from here. Extract the PHP file in the archive you've just downloaded.

Next up, you'll need the ZIP version of the Joomla! patch, which you can download from the Joomla! project's file repository site. Depending on which Joomla! version you've already installed, download the appropriate ZIP file. The tar.gz file will not do! For example, if you've got Joomla! 1.5.6 and want to update to 1.5.7 you must download the Joomla_1.5.6_to_1.5.7-Stable-Patch_Package.zip.

Upload the Joomla! ZIP file and kickstart.php to your site's root. If you used FTP, make sure both files' permissions are 0777. Make sure you have a working backup of your site before you proceed! In the unlikely event you run into a problem we want to be sure you can roll back. There are many ways to take a backup of your site, but we obviously suggest you to use the open source JoomlaPack backup component.

Now visit the kickstart.php URL, which is something like http://www.example.com/kickstart.php, substituting www.example.com with the full URL to your Joomla! site. In the page presented to you, make sure the update ZIP is selected in the topmost combo box. For the operation method we suggest using the "JavaScript Redirects" method, unless you know what you're doing. Depending on your site configuration you can either use "Write directly to files" or "Use FTP", supplying your FTP connection information as well. Most users will need the second (FTP) option.

After you're done setting up Kickstart, hit the big green "Start" button. You will see your browser reloading the page many times; this is normal. On the final page, click the second link titled "here". As the page reads, you're (almost) done! If you had an .htaccess file on your site, it is now renamed to htaccess.bak. Using an FTP client, rename it back to .htaccess and you're really done, in - virtually - no more than a few seconds!

Try it! I just upgraded two sites in 3 minutes flat (including upload times!).

Editor's Pick and site look

JoomlaPack is the new Joomla! Editor's pick for extensions. We are greatly honored to be chosen. An official announcement of this will be made later on the site.

JoomlaPack has a new site template. The template was designed by JoomlaPraise and it is their Joomla! 1.5 template contest entry. We chose this template because of its clean design and its emphasis on content. There still are some minor problems with the forum version of this template so we're going to give it one last shot before we release it as is. As with anything that changes, things may go wrong so please do tell us on the forums any problems you see with the design.

The upcoming JoomlaPack 1.3 brings new features

We might be just a few days before the release of 1.2.1 stable, but the JoomlaPack development team doesn't call it a day. We are actively developing the upcoming minor number version of the component. As with all past minor number versions, it brings along many new features. We feel it's only fair to share with you.

• Joomla! 1.5.x native mode only. If you follow Joomla! announcements you should have known - since about 8 months - that Joomla! 1.0.x development has stopped. No further updates are planned for that version of our favourite CMS. In other words, Joomla! 1.0.x is dead. Period. Our single most major step for JoomlaPack 1.3 was removing J! 1.0.x support and sticking with J! 1.5.x. Not constrained by the limitations of the old generation Joomla! and taking advantage of the empowering Joomla! Framework we have set to develop a modern code base, based on the principles of the MVC pattern, which will be much more easily maintenable in the future.
• Backup profiles. Until now, JoomlaPack was constrained to a single configuration. Setting configuration parameters and filter options was of an all-or-nothing approach. If you wanted to tweak some filters to get a different type of backup (let's say, filtering out some ill-behaving component) and then revert to your regular settings it was impossible. JoomlaPack 1.3 incorporates backup profiles, an unlimited number of profiles storing configuration option and filter settings. What's more, you can select which profile to use at backup time, even with front-end backup!
• Backup statistics. You asked for and you get it. JoomlaPack 1.3 will keep a lot of information about each backup taken. This has some nice implications you're gonna love:
  • No more need to reset after failed backup attempts. Based on information from Backup Statistics, JoomlaPack's engine will detect failed backups upon visiting the Control Panel page or starting a new backup, cleaning after them.
  • Describe and comment on your backups. No more need to remember what changes were made between individual backups (applies to backups started from the backend)
  • Ability to keep a limited amount of backup archives on your output folder automatically.
  • More informative backup administration page.
• Optional non-AJAX mode for the administration. Up until now, if your configuration was incompatible with AJAX you wouldn't be able to use JoomlaPack. After all, it relied on AJAX to operate the filters pages. This is no longer true. All basic operations on the filters pages are now possible to be executed in non-AJAX mode. Just select "JavaScript Redirects" for the "Backup Mode" in the Advanced tab of the configuration. The non-AJAX mode does not apply to some more advanced operations, like checking for database connectivity while editing a Multiple Databases setting, or fetching the default output directory in the configuration page.
• No more temporary directory setting. Joomla! 1.5.x provides with a standard temporary files location and JoomlaPack uses it. Of course, it needs to be writable!
• Embedded documentation. The basic set of documentation (help for each JoomlaPack page) will be included with the component, available by pressing the "help" button on the toolbar.
• Automatic problem detection. There are a number of server configurations which might cause JoomlaPack to fail. We know about them and know how to detect them. The next big step is warning you in advance, right from the Control Panel! It's just like the "You're using the default output directory" warning of JoomlaPack 1.2.1, only it covers a whole lot more situations.
  
• Revamped Kickstart. The new Kickstart will have a few changes over its predecessor. For starters, there will be no more two different files for AJAX and non-AJAX mode. The mode selection will be a radio button on Kickstart's first page. We are also planning to implement a FTP mode, to get past servers using Safe Mode and/or have "wrong" permissions.
  

Roadmap to JoomlaPack 1.2.1

Just over a week ago the stable 1.2 version was released to the public. As always, the stable release signifies the beggining of a new development cycle. This time, instead of jumping straight to developing version 1.3, we decided to improve the latest release in order to produce a safer, more performing, mature component.

This is the list of features we are currently working on, for version 1.2.1:

• Administrator directory restructuring and change of class names. CUBE is turning to a framework of its own right, built atop the Joomla! Framework. A big step towards full Joomla! 1.5 framework adoption, much of the guts of our component is being recoded to use only J! 1.5 API calls, while providing a backport layer for J! 1.0.x compatibility.
• Reduce and concatenate the steps (domains) needed to produce a backup. It occured to me that only two domains are actually required and should be present: database backup and file backup. In order to achieve this, a number of changes have to take place:
• Avoid extracting the installer files to temporary directory. Use the packaged installer files to "seed" the backup file instead. This also has a security enhancement bonus, as no "live" PHP files are written to the temporary directory, eradicating a direct access attack exploiting such files.
• Use name mangling on temporary files to minimize possibility of direct file access attacks
• Store the temporary file names and use this table when cleaning up. Moreover, clean up each temporary file right after it's been put in the archive. This is much more reliable than the current method which almost always leaves leftovers.
• The file list creation and packing steps are merged. This also means that much less data (in number of SQL queries and in total data size alike) is required to be written to database, solving many errors related to db server overload.
• Drop the "fast" algorithm. It is meaningless on most servers (crashes with timeouts) and the "smart" algorithm is already fast enough.
• Introduce Magic Numbers for smart algorithm. This will let you fine-tune the smart algorithm's performance to accomodate for slow, overloaded servers.
• Re-implement translations. It serves a double purpose: get closer to J! 1.5 compatibility and allow the use of semi-automated translation tools on the translation INI files. Some highlights:
• Introduce upercase INI keys by concatenating header and key, inserting an underscore between them
• Separate front-end and back-end language files
• Make a static method CLangManager::_($key) which translates the string like JText::_($key)
• Error and warning propagation and handling. Right now, if an error occurs in some part of the backup engine, it is not propagated and the process crashes in an unrelated point or completes to a partial backup (or, worse, no backup at all!). The idea is that if an error occurs anywhere in the code, the execution stops immediately, the error message is displayed to the user and also gets logged. In the case of warnings, they will be displayed just below the backup process messages.
• Distributed AJAX handling. The AJAX proxy will be taken apart and AJAX handlers will be grouped by page, hopefully creating less overhead during the actual backup process.
• JoomlaPack Installer 3. We are planning on creating the third iteration of our smart restoration script. This time it will be J!1.5-specific, based on the original J! 1.5 installer application. This is much harder than the traditional approach we took with JPI and JPI2, but it will be more forgiving in the case of errors - as you well know JPI2 is notorious for crashing silent when an error occurs during a J! 1.5 site's restoration. It will also cater for changing the cache and temp directories to their defaults if the pre-configured directories do not exist or are not writable, fulfilling a frequently made feature suggestion.

These are quite a mouthfull for a sub-minor number release! Right now the bulk of heavy refactoring is complete, namely the first two points. The rest of them are quite easy to implement, with the exception of the last two points which are bound to consume lots of our time.

The planned release date for 1.2.1 stable is mid-October 2008. Stay tuned!