ant

Ants are eusocial insects of the family Formicidae /fɔrˈmɪsɨdiː/ and, along with the related wasps and bees, belong to the order Hymenoptera. read more at WikiPedia



  • Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage. Martin Fowler about continuous integration

    Things get clearer in my mind...I will use a set of Open Sources software to reach this ambitious goal.

    Brainstorming

    Maintain a Single Source Repository.

    Either Subversion running locally on windows/Linux, but I will stay with www.joomlaCode.org at the beginning

    Automate the Build

    • I am currently evaluation Maven for PHP but already have ANT script to build components and plugin for Joomla 1.5 (they are not generic enough at the moment)

    Make Your Build Self-Testing

    • PHP UNIT tests if available with be run at this stage using PHP command line.
    • Seleniumto automate components testing across many platforms, as it runs in many browsers and operating systems, and can be controlled by many programming languages and testing frameworks.
    • Some standard utilities to perform standard Joomla! processes: login, logout for Apache ANT or Selenium
    • I will develop either a ANT plugin or a set of Selenium test cases for deploying and removing
      • component: install, publish, remove
      • plugin: install, publish, remove
    • All these utilities will be available under GPL v3 and as such free to reuse, and improve.

    Everyone Commits Every Day

    • This is reserved to developer having a www.joomlaCode.org account and part of the development team.

    Every Commit Should Build the Mainline on an Integration Machine

    I will use TeamCity continuous build server as it is free for personal use (limited to 20 concurrent build), run on Windows and Linux but require a java VM to run (may not be wanted in a PHP environment). Anyway with ANT, it will be possible to use another build server like Cruise Control or PHP Under Control.

    Keep the Build Fast

    That is an objective :-)

    Test in a Clone of the Production Environment

    Joomla! Build farm

    • I can imagine a set of Joomla instances, ideally 5 of each version, aka Joomla! 1.5.3 to Joomla! 1.5.8 and Joomla! 1.0.10 to 1.0.15
    • Joomla instances will be recreated at build time (files and databases), that mean Joomla! will get newly installed and removed in case of successful build
    • All Joomla! instances will be running with XAMPP, ideally on port not available to the outside world for security reasons

    Make it Easy for Anyone to Get the Latest Executable

    Successful build (Artifacts) are only available if build is successful. Team City provide this with less effort (configuration)

    Everyone can see what's happening

    • A guest account will be available or a free public area with limited access to see the result of builds.
    • RSS feeds, emails and Instant messaging (Jabber) out of the box for end users or developers!

    Automate Deployment

    That will be, auto publish to some demo site in a configurable way. At the moment, at http://demo.waltercedric.com and http://demo2.waltercedric.com for me :-)

    Final words

    • I will provide a ready to use package for Windows and Linux and all scripts, so anybody will be able to run it also on your own.
    • Critical part will be documented in my WIKI at http://wiki.waltercedric.com direct link HERE

    It seem that nobody is providing such a package as I am after only one day at the top of search results in Google "continuous build joomla"

  • Automatic installation of Joomla! runtime environments

    Main ideas

    Build is scalable

    Distributed build management optimize hardware resources utilization by parallelizing product builds within the build agents grid. With build chains support, it is even possible to break down a single build procedure into several parts to run them on different build agents &8212; both in sequence and in parallel &8212; using the same set of sources in all of them.

    I want to be able to test my components against many versions of Joomla!

    All versions of Joomla! are in subversion as zip files in an own SVN repository

    For example:

    • ${JOOMLA15_VCS_ROOT}&160; is svn:\\localhost\joomla1.5\trunk
    • ${JOOMLA10_VCS_ROOT}&160; is svn:\\localhost\joomla1.0\trunk

    These repository ${JOOMLAxx_VCS_ROOT} are connected to all build as supplementary VCS root in TeamCity and thus content get checked out as part of the build in the build temporary directory of one agent. ($AGENT_BUILD_DIR)

    joomla1.5\trunk
    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.4-Stable-Full_Package.zip
    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.5-Stable-Full_Package.zip
    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.6-Stable-Full_Package.zip
    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.7-Stable-Full_Package.zip
    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.8-Stable-Full_Package.zip

    So after the checkout, the file system will look like

    ($AGENT_BUILD_DIR)\&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;

    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.4-Stable-Full_Package.zip
    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.5-Stable-Full_Package.zip
    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.6-Stable-Full_Package.zip
    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.7-Stable-Full_Package.zip
    &160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160;&160; Joomla_1.5.8-Stable-Full_Package.zip

    If you don't want to provide support a a specific version of Joomla! just remove it from the trunk! or add new ones on purpose. That's easy.

    Ant tasks/Maven MOJO

    • Are responsible for unpacking all these zip files to the build temporary agent directory. ($AGENT_BUILD_DIR).
    • Filenames are found with a configurable regular expression,
    • All settings will be committed in \joomla1.5\trunk\build.deploy.properties

    Another ant script/task will for each zip,

    • Start a Selenium test cases that will create a virtual user that use the regular Joomla! installer and drive installation till the end.
    • All settings which have to be Joomla! and build independent will be randomly generated, preferably UUID for password and database name for example.
    • Login and Admin password may be the same (admin:admin) at the beginning but can also be generated and written to a file on disk in ($BUILD_DIR)/{joomlarootversion}/build.install.properties.
    • Directory installation ($BUILD_DIR)/{joomlarootversion}/installation will be renamed to ($BUILD_DIR)/{joomlarootversion}/installation.old or simply deleted
    • Selenium/PHP Unit test that are committed in \joomla1.5\trunk\Installation.Checks will perform basic checks (login, logout, navigate) to ensure that installation of Joomla! has been successful.
      If everything succeed, we will have a set of Joomla! versions ready for our components regression testing.

    Remarks:

    • No build temporary directory. ($AGENT_BUILD_DIR) will be deleted by Ant or Maven but by the build server itself. This will let developer look at the issues on file system and in database.
    • New scripts may be developed to extract from the build server or Joomla! farm easily the non running Joomla! instance files + database) so developers can install the broken setup locally.

    Automatic deployment of Joomla! components

    Your component is typically shared and many developer committed regularly in a different VCS root... For SecurityImages 5.x.y, subversion root may be&160; svn:\\localhost\securityimages5\trunk

    This VCS root is also attached to the build and get check out at build time by TeamCity.

    Packaging

    if a build.xml is present in {VCS_ROOT}\build.xml then it is executed prior to any further operations, purpose of build.xml is to produce a component binary distribution (zip or tar.gz) that can be then installed to ALL Joomla install in the agent root directory.

    Deployment

    if a deploy.xml is present in {VCS_ROOT}\deploy.xml then it is executed, purpose of deploy.xml is to deploy one or many component binary distribution (zip or tar.gz) to ALL Joomla install in the agent root directory.

    Why one or many component?

    I want to be able to track also component dependencies issues.

    Lets say that SecurityImages does not play well with VirtueMart, I may want to test also that combination across Joomla! instances, that's why VirtueMart may have to be deployed with SecurityImages or not.

    prerequisites:

    • Running SVN server, see HERE for installing it on windows
    • Installed JVM, latest JDK 1.6u10
    • Running TeamCity server
    • Running XAMPP with HTTP root directory at TeamCity agent root directory.
    • Apache ANT with additional library for more control (if, case, for loop)

    This articles will be available in my WIKI soon http://wiki.waltercedric.com/index.php?title=ContinuousBuildforJoomla so any reader or developer can participate to the discussion, next step is to implement the above and that will e documented as well :-)

  • I will commit soon a first draft (alpha) of what is expected to bring continuous build to any Joomla! component (or event to Joomla! core itself ;-))

    You are free to join the project, all documentation effort stay at the moment in my WIKI

    &160;

  • Call task1, if the variable contains 'true', 'on', or '1' the task 'send a sms alert' is executed
    
    
    
    
    
    
  • A properties file in c:/confA commentstmp.path=c:\tempftp.login.default=cedricwalterftp.pwd.default=xxxxftp.host=ftp.adress.comftp.port=21
    
    
    
    
    
    
  • This script ask for security reason the login and password. (but accept default values)
    
    
     
    
    
    
     
    
    
    
    
    
    
  • This script ask for security reason the login and password. (but accept default values)
    
      
    
    
    
    
    
    
    
    
    
    
    
    
    
    
  • I put some effort the last few days in this new framework.

    Done:

    • I did document some part of it at http://wiki.waltercedric.com/index.php?title=ContinuousBuildforJoomla
    • TeamCity is installed/configured/documented (windows only)
    • XAMPP is installed/configured/documented (windows only)
    • I also at the same time configured a Puppy Linux VMWare image that will permit to anybody to have a running environment in no time.
    • I am able to unpack all Joomla! versions that are committed in any repository (CVS, SVN, Clearcase)
    • They can be unpacked anywhere on file system (config in setEnv.xml), ideally they are unpacked in the root htdocs of XAMPP
    • Code is committed at Joomla forge in SVN http://joomlacode.org/gf/project/continbuild4j/

    Issues

    Selenium test suite do not accept a baseurl (or only a part of it) so I have a full path like /Joomla1.5.7/installation/index.php in a selenium test case instead of /installation/index.php in test case and baseurl= http://localhost/Joomla1.5.7)

    Architecture

    3rd Party

    • I use antelope for some advance ANT operations: substring, indexof, loop
    • I use selenium java server on port 4444 (default)

    Cluster

    All cluster operations are in cluster.xml these basic functions are

    • cluster.init
      • cluster.remove&160;&160;&160;&160;&160;&160;&160; remove all instances of Joomla! in checkout directory
      • joomla.unpack.all&160;&160;&160; unpack all Joomla! versions that are checked in another SVN root
      • joomla.install.all&160;&160;&160;&160;&160; run the same selenium test case joomla.install.html on all Joomla! instance
      • joomla.remove.all.installation&160;&160; remove all Joomla! installation directories
      • joomla.check.all&160;&160;&160;&160; check all Joomla! installations for correctness
    • cluster.start
    • cluster.remove
    • cluster.stop

    Joomla!

    All Joomla specific operations are in joomla.library.xml

    • Unpack a Joomla! version
    • Remove the installation directory form a version
    • Apply a selenium test suite install.joomla.html that can use the regular Joomla! installer
    • Is also able to do all of the above on all Joomla! versions found (regular expression) in checkout directory

    Selenium

    • All selenium operations are in selenium.xml
    • All test suite and test cases are in /selenium/joomla/

    PHPUnit

    All PHPUnit operation are in phpunit.xml

    Settings

    Settings are in setEnv.xml, in future I will lazy load a file if it exist as environment variable

    &160;

    If you know ANT, the code is quite readable...

  • apache_maven

    build.tools

     

     

     

     

     

    Even if the sample is quite small (600 responses), it still interesting going through these compiled data

    Some time ago we ran a survey asking a few questions about the build process, specifically the tools that are used to do incremental builds and how much time those builds take. We had over 600 responses, so now it’s time to count the results.

    This is the first time that we’ve published results on the incremental build process, so the information is more likely to serve as a guide than an authoritative information source.

    Read More here

  • Here is 3 different way to control the lifetime a local Tomcat 7 container using Apache Maven. A typical scenario would be to start a servlet container prior to running integration tests (Selenium, SAHI or using any other framework you can think of )

    With the following examples, you will be able to start an instance of Tomcat 7 running your web application in the pre-integration-test phase and stop the instance in the post-integration-test phase. You can also decide to use an embedded container like Jetty instead.