Tag: taskqueue

Posted on

Using Google Protobuf Timestamp In PHP

This is more of a documentary post because I haven’t seen documentation on Google’s Timestamp class anywhere.

Google’s libraries – in particular, the GCP libraries for datastore/tasks/etc – use the Google/Protobuf/Timestamp class to represent time. Timestamp is a simple wrapper around the number of seconds since UNIX epoch, in the UTC timezone. For example here is how to create a Timestamp reflecting the current date and time, plus 2 minutes into the future (120 seconds):

use Google\Protobuf\Timestamp;

    $future_time_seconds = time() + 120;
    $future_timestamp = new Timestamp();
    $future_timestamp->setSeconds($future_time_seconds);
    $future_timestamp->setNanos(0);

There are equivalent classes and functions for Python/Java/Go/other languages that Google Cloud supports.

Using the Timestamp class – especially setting up future dates – is necessary for configuring my favorite Google Cloud service: Cloud Tasks. A Cloud Task can accept a future date to run at, thereby giving you a way to queue up and delay execution of an activity. For example, see the below screenshot: I’ve created 3 tasks 20 seconds ago, yet they’re set for a future execution 3 minutes 45 seconds from now:

Posted on

RetryOptions Example In Java

Here’s a demonstration of how to set the retry options on a task. The following code configures a task to retry 3 times (if the initial request fails) and to double the time between retries at most twice.

//Configures the retry options for the task. 
//Here we're saying to retry 3 times if the task initially 
//fails, and to increase the time between retries.
RetryOptions retry = RetryOptions.Builder.withTaskRetryLimit(3);
retry.maxDoublings(2);

The following line sets the options onto a task ( task represents a TaskOptions object ):

task.retryOptions(retry);

Remember to import the RetryOptions class:

import com.google.appengine.api.taskqueue.RetryOptions;
Posted on

Task Queue Generating TransientFailureException

Task queue operations may rarely generate a TransientFailureException, which will look similar to this:

com.google.appengine.api.taskqueue.TransientFailureException:
    at com.google.appengine.api.taskqueue.QueueApiHelper.translateError(QueueApiHelper.java:106)
    at com.google.appengine.api.taskqueue.QueueApiHelper.translateError(QueueApiHelper.java:153)

In general this exception indicates a temporary issue with the underlying App Engine infrastructure, not a failure of the application code. This problem should fix itself shortly.

To mitigate this exception, catch it with a try/catch block and attempt to requeue the task. If that fails, insert a delay of a few seconds before attempting to requeue the task. Redeploying the application may also help.

Posted on

Sleeping An Application In Java

Some applications need to pause execution for a short amount of time. In App Engine, there are two ways to do that.

For short pauses, most programming languages have the concept of sleep operations. Here’s how to sleep for 2,000 milliseconds (2 seconds) in Java:

try {
    Thread.sleep(2000);
}
catch (InterruptedException e) {
}

If you want to pause the application for a few seconds, using the sleep function is a good idea. But if you need a longer pause – more than a minute or so – it’s a better idea to use the Task Queue’s countdownMillis function.

Here’s how to create a task that will wait for 5 minutes, then launch off another request:

TaskOptions task = TaskOptions.Builder.withCountdownMillis(5 * 60 * 1000);

In short, whenever you need a long pause in execution, queue up a task with a countdownMillis() delay. Then the task can call a request handler which can continue any application logic needed.

Posted on

Basic Java Task Queue Code

Here’s a simple example of how to use the task queue in Java. The code below retrieves the default queue and queues up a task. The task will request the /example_url path and pass in the parameter parameter1 with the value parameter1_value.

Queue queue = QueueFactory.getDefaultQueue();
TaskOptions task = TaskOptions.Builder.withUrl("/example_url");
task.param("parameter1", parameter1_value);
queue.add(task);

Remember to import the task queue classes:

import com.google.appengine.api.taskqueue.Queue;
import com.google.appengine.api.taskqueue.QueueFactory;
import com.google.appengine.api.taskqueue.TaskOptions;
Posted on

Inspecting Task Queue Logs

Here’s a sample view of a request from the Task Queue service. The log includes the name of the queue (in this case default), the task’s name (the long number starting with 1636460), and the instance that handled the request (the blue text on the bottom).

An important note: a quick and easy way to figure out if a request comes from the Task Queue is to look at the request’s IP address. All Task Queue requests come from the IP 0.1.0.2.

Posted on

TaskTooLargeError

Google App Engine has an upper limit on how much data you can put into a task – a maximum of 100 KB. This includes everything within an individual task such as parameters, payload, the request URL, etc.

To get around this error, store any large pieces of information within an entity in the datastore. You can then pass the entity’s ID as a task parameter and pull out the entity’s data during task processing.