How to externalize Magento cronjobs to filesystem crontab

How to externalize Magento cronjobs to filesystem crontab

Recently I’ve been working on a project where the product’s data was imported from an ERP system by a cron in Magento. Everything was OK until one day, when a new module was installed.

It had a cron of its own that processed the whole product collection and ended up disrupting Magento’s cron scheduling. This was a very critical issue in my case, as price and stocks were not updated and other important tasks were not performed by the cronjobs on the production environment.

The problems kept piling up fast, with products that didn’t get imported or updated at all. I had to find a way to solve this problem ASAP.

I really had to learn the hard way how to manage this kind of issues, but this doesn’t have to be the same for you. You can predict this behavior, optimize your technical solutions and tune your cronjobs in a way that such critical situations will be avoided.

I had to find a solution and fast!

First, we need to understand what happened. When a scheduled task runs and performs processing on large volumes of data, it may consume all the resources available, which will cause the other crons to be ignored by the scheduler or to be declared as "missed" by Magento. The result - they won’t be able to run at all.

So, for all you developers out there struggling with Magento crons, I hope my solution helps.

The idea was to decouple Magento crons so that they won’t depend on each other’s run. We want to ensure that Magento cronjob tasks are instantiated by the OS cron, not by Magento, so that multiple crons can be run independently from Magento’s cronjob tasks scheduling.This means that we need to implement a way to register these cronjobs inside the filesystem crontab.

So how does Magento cron work after all?

All cronjobs declared into modules config.xml files are not actual filesystem cronjobs. They are, in fact, tasks scheduled and instantiated by this /cron.php script, which is the only Magento cronjob registered into the filesystem’s crontab by default.

When Magento cron manager works, it stores the tasks’ activity in the "cron_schedule" table by registering the scheduled time, the execution duration, the status and eventual messages. This table is managed by the "Mage_Cron" module and will register the actions performed on the configured crons of other modules.

Now, if you take out the task from Magento’s native cron manager, you will lose the ability to track your cronjobs, as their status won’t be available in the "cron_schedule" table.

Moreover, if you are using cron monitoring extensions like Fabrizio Branca's AOE_Scheduler (which I used at that point to monitor the crons), you won’t be able to filter and search the cron’s reports in the admin area, as it won’t find your cron.

But what if we can trick Magento into "thinking" that it has a cronjob of its own, when it actually doesn’t?

What I did was declare the cronjob in my module’s config.xml file, like I normally do, but this time with a little tweak: the schedule expression looks like this: * * 30 2 *” - which is a fictive date of course, but we want Magento to be able to find our cronjob when filtering the cronjobs collection, although not to actually execute it.

I’m assuming that you already know how to create a new module or that you have a module with a custom cronjob declared, so go to the "config.xml" file, adapt your cronjob configuration and make it look something like this:


  
  
    
      
        * * 30 2 *
        
          evozon_cron/observer::dummyRun
        
      
    
      

Please note the cron expression and the method being called. I created a dummy method just for the sake of a valid configuration (the method won’t be called at all due to the schedule expression).

A little bit of change here and there

In your module, modify the cron method from the class that is responsible for task processing and manually register the cronjob’s status at the start and the end of the processing. In my case:

<?php

class Evozon_Cron_Model_Observer
{
  /**
    *  Perform the products data import
    */
    public function importProductsData()
    {
      // save the cronjob running status
      $messages = "Cron is running";
      $timeStart = strftime('%Y-%m-%d %H:%M:%S', time());
        
      /** @var $cronModel Mage_Cron_Model_Schedule */
      $cronModel = Mage::getModel("cron/schedule");
      $cronModel->setJobCode("evozon_import_products")
        ->setStatus(Mage_Cron_Model_Schedule::STATUS_RUNNING)
        ->setCreatedAt($timeStart)
        ->setScheduledAt($timeStart)
        ->setExecutedAt($timeStart)
        ->setMessages($messages)
        ->save();        
        
      try {
            
      /**
        * .....
        * perform the task processing
        * .....
        */
            
        $messages = "Cron run successful";
        $finalStatus = Mage_Cron_Model_Schedule::STATUS_SUCCESS;
                  
      } catch (Exception $e) {
        Mage::logException($e);
        $messages = $e->getMessage();
        $finalStatus = Mage_Cron_Model_Schedule::STATUS_ERROR;
      }
        
      if ($cronModel->getId()) {
        $timeEnd = strftime('%Y-%m-%d %H:%M:%S', time());
        $cronModel->setStatus($finalStatus)
          ->setFinishedAt($timeEnd)
          ->setMessages($messages)
          ->save();
      }
    }
    
    /**
      *  Dummy method that will never be executed
      *  but needed for a valid cron configuration
      */
    public function dummyRun()
    {
      return;
    }    
}

Now, you need to add a script that will act as an entry point for your cronjob when being executed from outside Magento. What I did was simply create a new directory called "crons" inside the /shell/ directory (for organizing purposes, you can place your script directly in the shell directory if you like) and add a simple script that will call the class and method that you need to do the processing:

<?php

require_once __DIR__ . '/../abstract.php';
class Evozon_Shell_Cron_Import_Products extends Mage_Shell_Abstract
{
/**
  *  Call cron task
  */ 
  public function run()
  {
    $observer = Mage::getModel("evozon_cron/observer");
    $observer->importProductsData();
  }
}
$script = new Evozon_Shell_Cron_Import_Products();
$script->run();

Externalize your cron

Now you need to register your cronjob into the filesystem’s crontab. First, you need to find the path to your PHP binary, because the cron is going to execute a ".php" script. If your Magento cron is registered in crontab with the "cron.php", you can take the PHP binary location from there. If it’s the "cron.sh" that triggers Magento cron tasks and you don’t know for sure where your PHP installation is, you can find it by running:

which php

In my case, the location is:

/usr/bin/php

Even though Magento’s "cron.php" cron could be executed from "root" crontab, I personally do not recommend that you run your custom crons as root.

Instead, you could register your crons under the web-server filesystem user crontab. In my case, I was using "nginx". So, from the /etc/nginx/nginx.conf file, using the "user" directive, I found the web-server user as "www-data" (which is also the user for apache web-server). Check if this is the case for you too. If not, write your web-server user instead of "www-data" in the following command:

sudo -u www-data crontab -e

then add a new line that will contain the running schedule, the path to php installation you previously found and the path to your script (with absolute path to your Magento installation). Mine looks like this:

00 01 * * * /usr/bin/php /var/www/html/magento/shell/crons/products.php

Another way to monitor crons

If you are not using a custom extension that will allow you to see the cron monitoring in the shop’s administration and you don’t like querying the database (or you don’t have access to it), you can also provide output from your cron method with simple ‘echo’s and monitor each cron’s steps in a separate log. So, for example, assuming you have 2 cronjobs: "products.php" and "categories.php", you can configure the crontab tasks like this:

00 01 * * * /usr/bin/php //shell/crons/products.php >> //var/log/cron_products.log 2>&1
30 02 * * * /usr/bin/php //shell/crons/categories.php  >> //var/log/cron_categories.log 2>&1

This configuration will allow you to see messages from both stderr and stdout in the same file. You can also use "Mage::log()" method to log your debug lines and redirect only the stderr in the same file as your debug log. It’s a matter of preference here.

You don’t need to restart any service at all, as the system will register the change in the crontab as soon as you save the file and you will get a message like this:

crontab: installing new crontab

From panic to relief

The problem originated in a non-optimal solution provided by an extension that ended up disrupting Magento’s cron activity. It’s true that you cannot control the code written by others, especially when the decision on installing an extension is not yours to make. But, as a best practice, while developing your solutions, keep in mind the impact it might have on the system and try to optimize your implementation as much as you can.

In some cases (such as mine), you can offer a bit of a relief to Magento’s cron manager if you externalize your custom crons. Soon, you will notice that your Magento cron runs smoothly, as all it has to do is schedule and run its own native tasks. No more missed or ignored scheduled tasks. Your cronjobs run independently from Magento’s native cron manager and you are still able to monitor their activity.

Of course, this was a simple way to cope with problematic cronjobs. I am sure you guys found other solutions too and I invite you to share your approach in the comments section below.


NO COMMENTS

Tell us what you think

Fields marked with " * " are mandatory.

We use cookies to offer you the best experience on our website. Learn more

Got it