Retweeting In Java

Earlier I posted examples of how to log into Twitter and post a tweet using the twitter4j library. Now here’s a function demonstrating how to retweet a post.

This function is entirely self-contained; all it requires is a global variable twitter, which represents a twitter4j.Twitter object preconfigured with the proper authentication details.

public void doRetweet() {
    /**
     * To demonstrate retweeting, we'll search Twitter for all tweets that contain 
     * the phrase "mail not working" (including quotation marks). We'll then select 
     * a random tweet, and retweet it into our stream.
     */
    try {
        //Search for tweets, then pull out a list of those tweets.
        //A Status is how twitter4j refers to an individual tweet.
        twitter4j.Query q = new twitter4j.Query("\"mail not working\"");
        q.count(100);
        List<Status> results = twitter.search(q).getTweets();
        //Randomly select a tweet
        Random generator = new Random();
        int pick = generator.nextInt(results.size());
        Status status_to_retweet = results.get(pick);
        //Log the tweet we're retweeting
        System.out.println("Now retweeting: " + status_to_retweet.toString());
        //And finally retweet that tweet
        long status_id = status_to_retweet.getId();
        twitter.retweetStatus(status_id);
    }
    catch (TwitterException e) {
        System.err.println("A TwitterException has occurred while attempting to retweet. Exception message: " + e.getMessage());
    }
}//end function

Extracting The Subdomain In Java

A short code example today: how to extract the subdomain in Java. The req object represents HttpServletRequest.

String subdomain = req.getRequestURL().toString();
subdomain = subdomain.substring(subdomain.indexOf("/") + 2, subdomain.indexOf("."));

For example, if the user entered http://subdomain.example.com , this code would store the word subdomain in the subdomain variable.

Displaying Time In Java

Web applications frequently need to display times and dates in other timezones, not just their local time zone or UTC. Here’s a code example that takes a date and expresses it in several different time zones:

Date add_date = new Date();
SimpleDateFormat formatter = new SimpleDateFormat("MMMMMMMMMM dd, yyyy hh:mm aa");
formatter.setTimeZone(java.util.TimeZone.getTimeZone("America/Los_Angeles"));
String cali_time = formatter.format(add_date);
formatter.setTimeZone(java.util.TimeZone.getTimeZone("America/Denver"));
String denver_time = formatter.format(add_date);
formatter.setTimeZone(java.util.TimeZone.getTimeZone("America/Chicago"));
String chicago_time = formatter.format(add_date);
formatter.setTimeZone(java.util.TimeZone.getTimeZone("America/New_York"));
String ny_time = formatter.format(add_date);
formatter.setTimeZone(java.util.TimeZone.getTimeZone("France/Paris"));
String paris_time = formatter.format(add_date);
formatter.setTimeZone(java.util.TimeZone.getTimeZone("Russia/Moscow"));
String moscow_time = formatter.format(add_date);
formatter.setTimeZone(java.util.TimeZone.getTimeZone("China/Shanghai"));
String shanghai_time = formatter.format(add_date);

Google Cloud Storage RetryHelper Failure

I have numerous applications which use Google Cloud Storage buckets to save files, maintain backups, etc. Most of these applications are Java apps using the official Google Cloud Storage library.

Yesterday one of these applications had difficulty accessing Cloud Storage and printed out the following error report:

The GCS library attempted to access Cloud Storage a total of 6 times; this image shows the last failure and the final “giving-up” error.

Here’s a copy of the error in text form:

com.google.appengine.tools.cloudstorage.RetryHelper doRetry: 
    RetryHelper(4.114 s, 6 attempts, com.google.appengine.tools.cloudstorage.GcsServiceImpl): 
    Attempt 6 failed, sleeping for 3140 ms: java.io.IOException: Response code 503, 
    retryable: Request: POST https://storage.googleapis.com/[bucket_url]/[file_name]
x-goog-resumable: start
x-goog-api-version: 2
Content-Type: image/jpg

no content

Response: 503 with 19 bytes of content
Content-Length: 19
Date: Thu, 01 Aug 2013 [time]
Server: HTTP Upload Server Built on Jul 21 2013 19:20:38 (1374459638)
Content-Type: text/html; charset=UTF-8
X-Google-Cache-Control: remote-fetch
Via: HTTP/1.1 GWA
Service Unavailable

And after 6 failures attempting to access the Google Cloud Storage bucket, the library reports an error and stops:

.<stderr>: IO Exception: RetryHelper(4.115 s, 6 attempts, 
    com.google.appengine.tools.cloudstorage.GcsServiceImpl): 
    Too many failures, giving up

Fortunately this seemed to be only a transient problem; the next upload to GCS succeeded.

Generate SHA1 Hash

Many web applications need to generate a SHA1 hash for verifying file integrity. If you use the Apache Commons libraries, the DigestUtils class offers convenience methods for generating hashes.

If you don’t use Apache Commons, the following function will generate SHA1 hashes for you. Simply pass the string to calculate the hash from into the function ( to_be_sha1 ); the function will then return the SHA1 hash. If an error is encountered (such as the passed-in string being null), a RuntimeException will be thrown.

/**
 * Calculates a SHA1 hash from a provided String. If 
 * to_be_sha1 is null, a RuntimeException will be thrown.
 * 
 * @param to_be_sha1 String to calculate a SHA1 hash from.
 * @return A SHA1 hash from the provided String.
 */
public static String generateSHA1(String to_be_sha1) {
    String sha1_sum = "";
    //if the provided String is null, throw an Exception.
    if (to_be_sha1 == null) {
        throw new RuntimeException("There is no String to calculate a SHA1 hash from.");
    }
    try {
        MessageDigest digest = MessageDigest.getInstance("SHA1");
        byte[] array = digest.digest(to_be_sha1.getBytes("UTF-8"));
        StringBuffer collector = new StringBuffer();
        for (int i = 0; i < array.length; i++) {
            collector.append(Integer.toString((array[i] & 0xff) + 0x100, 16).substring(1));
        }
        sha1_sum = collector.toString();
    }//end try
    catch (NoSuchAlgorithmException e) {
        throw new RuntimeException("Could not find a SHA1 instance: " + e.getMessage());
    }
    catch (UnsupportedEncodingException e) {
        throw new RuntimeException("Could not translate UTF-8: " + e.getMessage());
    }
    return sha1_sum;
}//end generateSHA1

Add these imports as well:

import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;

Reading In A File With Java

Here’s a short code example demonstrating how to read in a file within your application. This snippet assumes the file data.json is within your application’s root directory (change as necessary). The contents of the file are read into the variable json – insert your own processing code after the while code block. If the file cannot be found, or if there are other difficulties in reading from the file, an IOException will be thrown.

try {
    InputStream is = context.getResourceAsStream("/data.json");
    BufferedReader reader = new BufferedReader(new InputStreamReader(is));  
    String json = "";
    String line = "";   
    while (line != null) {
        json += line;
        line = reader.readLine();
    }   
    //Do something with the read-in data within the json variable.
}
catch (IOException e) {
    throw new RuntimeException("Unable to read in file.");
}

Generating MD5 Hashes In Java

Web applications frequently need to generate hashes for comparing data, verifying file integrity, generating keys, etc. If your application uses the Apache Commons libraries, you can use the included DigestUtils convenience class to create hashes. If you don’t use the Apache libraries, then you need an alternate way to generate MD5 hashes.

The below code snippet is a simple function that generates a MD5 hash from a given string. It’s simple to use: simply pass the string to be hashed into the function (to_be_md5), and the hash is returned. This function will throw a RuntimeException if an error is encountered (for example, if the passed-in String is null).

/**
 * Calculate a MD5 hash from the provided String. If the 
 * provided String is null, this method will throw a 
 * RuntimeException.
 * 
 * @param to_be_md5 A String to calculate a MD5 hash from.
 * @return A MD5 hash calculated from the provided String.
 * @throws RuntimeException If an error was encountered during calculating.
 */
public static String generateMD5(String to_be_md5) {
    String md5_sum = "";
    //If the provided String is null, then throw an Exception.
    if (to_be_md5 == null) {
        throw new RuntimeException("There is no string to calculate a MD5 hash from.");
    }
    try {
        MessageDigest md = MessageDigest.getInstance("MD5");
        byte[] array = md.digest(to_be_md5.getBytes("UTF-8"));
        StringBuffer collector = new StringBuffer();
        for (int i = 0; i < array.length; ++i) {
            collector.append(Integer.toHexString((array[i] & 0xFF) | 0x100).substring(1, 3));
        }
        md5_sum = collector.toString();
    }//end try
    catch (NoSuchAlgorithmException e) {
        throw new RuntimeException("Could not find a MD5 instance: " + e.getMessage());
    }
    catch (UnsupportedEncodingException e) {
        throw new RuntimeException("Could not translate UTF-8: " + e.getMessage());
    }
    return md5_sum;
}//end generateMD5

Don’t forget to import the following classes:

import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;

Retrieving All Recently Created Entities

Searching the datastore using date-based entity properties can often be confusing to new App Engine developers. Here’s how to search for all recently created entities.

First, each entity has to record the time it was created. Add the following line of code when constructing your entities (entity represents the Entity object being created):

entity.setProperty("add_date", new Date());//Records the date this entity was added.

Here’s the code for the query. This code example searches for all entities created within the last 24 hours with a kind of mail. The resulting entities are placed into the variable results (which is a standard java.util.List object).

/**
 * Creates a Date object that represents a point in time 
 * 24 hours ago.
 */
Date current_date = new Date();//The current date.
long current_date_in_ms = current_date.getTime();//The current date in milliseconds.
//Take the current date in milliseconds, and subtract 24 hours worth 
//of milliseconds (1000 milliseconds per second, times 60 seconds per 
//minute, times 60 minutes per hour, times 24 hours in a day).
long one_day_ago_in_ms = current_date_in_ms - (1000 * 60 * 60 * 24);
Date one_day_ago = new Date(one_day_ago_in_ms);//Create a new Date object representing one day ago.
//Search all entities with the kind "mail". 
//We create a Filter that requires that the add_date property be greater 
//than the Date we computed above.
Query q = new Query("mail");
Query.Filter date_filter = new Query.FilterPredicate("add_date", Query.FilterOperator.GREATER_THAN_OR_EQUAL, one_day_ago);
q.setFilter(date_filter);
PreparedQuery pq = DatastoreServiceFactory.getDatastoreService().prepare(q);
List<Entity> results = pq.asList(FetchOptions.Builder.withDefaults());

 

Searching Twitter

Many web applications integrate Twitter into their pages to display realtime news and thoughts. Here’s a code snippet showing how to search Twitter using the twitter4j library.

Twitter’s search API doesn’t return all search results at once; instead it returns “pages” of results, with each page containing 100 tweets matching the search criteria. The following code snippet runs through many pages of search results using a while loop, so it may take some time to process. Run it within a backend if you’re searching for a particularly popular phrase or doing heavy processing on each tweet.

The variable twitter represents a twitter4j.Twitter object, and status represents a single tweet. You can extract tweet information from that object; for example status.getText() would return the text of the tweet. Edit the string “google app engine” to whatever text you’re searching Twitter for.

twitter4j.Query twitter_query = new twitter4j.Query("google app engine");
twitter_query.setCount(100);
//This while loop runs through each page of the returned 
//tweets. One page of results (100 tweets) is processed 
//per loop.
while (twitter_query != null) {
    //A list of the returned tweets, representing 1 page (100 tweets).
    QueryResult twitter_results = twitter.search(twitter_query);
    //Run through this page of results and access each returned tweet.
    for (Status status : twitter_results.getTweets()) {
        //Do something with the status object.
    }
    //Retrieves a Query representing the next page of results.
    twitter_query = twitter_results.nextQuery();
}//end while loop running through pages of returned results.

 

HTTP GET Using The Low Level Java App Engine API

Here’s a short code example showing how to do a HTTP GET using the low level Java API.

The variable url_string_here is the URL being retrieved as a String. It returns a byte[] array containing the content of the response. If the response code is not 200 (i.e. anything other than HTTP OK) then this code throws a RuntimeException.

URL url = new URL(url_string_here);
HTTPRequest request = new HTTPRequest(url, HTTPMethod.GET);
request.setHeader(new HTTPHeader("User-Agent", "Custom User Agent "));
//Execute request.
HTTPResponse response = URLFetchServiceFactory.getURLFetchService().fetch(request);
if (response.getResponseCode() == 200) {
    //The response was OK
    //Retrieve the content of the response.
    return response.getContent();
}//end if the response code was 200.
else {
    throw new RuntimeException("Response code was " + response.getResponseCode());
}