Tips, tricks, and thoughts about Google, AdWords, Google Cloud Platform, and all its subsidiaries. Not affiliated with or sponsored by Google.
I opened a new GCP project to host a Python application when I hit a problem – my logging.info() and logging.warn() statements weren’t showing up in my logs. Then I realized the standard error and standard out streams weren’t selected in logging!
If you’re missing log information, make sure to select the correct streams in the second dropdown box, as in below:
I use PhantomJSCloud to take screenshots of web apps. While writing new code, I noticed this error coming back from the server:
{
"name":"HttpStatusCodeException",
"statusCode":400,
"message":"Error extracting userRequest. innerException: JSON5: invalid character 'u' at 1:1",
"stack":[
"no stack unless env is DEV or TEST, or logLevel is DEBUG or TRACE"
]
}The problem came because the request wasn’t JSON-encoding the object; the fix looks like this (using the requests library):
post_data_string = json.dumps(python_object_here, sort_keys=True, indent=3, separators=(',', ': '))
r = requests.post('https://phantomjscloud.com/api/browser/v2/', data=post_data_string)This short code segment turns today’s date into a folder path. For example, today’s date of April 2, 2019 would be turned into 2019/04/02. I use it for categorizing date related documents into appropriate folders within GCS.
var datestamp_obj = new Date( (new Date()).getTime() + (-6) * 3600 * 1000 );
var datestamp_slash = datestamp_obj.getFullYear() + "/";
datestamp_slash += datestamp_obj.getMonth().toString().padStart(2, "0") + "/" + datestamp_obj.getDate().toString().padStart(2, "0");
This is more of a documentary post because I haven’t seen documentation on Google’s Timestamp class anywhere.
Google’s libraries – in particular, the GCP libraries for datastore/tasks/etc – use the Google/Protobuf/Timestamp class to represent time. Timestamp is a simple wrapper around the number of seconds since UNIX epoch, in the UTC timezone. For example here is how to create a Timestamp reflecting the current date and time, plus 2 minutes into the future (120 seconds):
use Google\Protobuf\Timestamp;
$future_time_seconds = time() + 120;
$future_timestamp = new Timestamp();
$future_timestamp->setSeconds($future_time_seconds);
$future_timestamp->setNanos(0);There are equivalent classes and functions for Python/Java/Go/other languages that Google Cloud supports.
Using the Timestamp class – especially setting up future dates – is necessary for configuring my favorite Google Cloud service: Cloud Tasks. A Cloud Task can accept a future date to run at, thereby giving you a way to queue up and delay execution of an activity. For example, see the below screenshot: I’ve created 3 tasks 20 seconds ago, yet they’re set for a future execution 3 minutes 45 seconds from now:
In a previous post, I posted a sample NodeJS function to assemble a WordPress blog slug. I ended up rewriting part of the larger application (and the function itself) in Python.
In the below function, source is a blog title string, and it returns a slug suitable for use in a blog URL.
def generate_slug(source):
i = 0
source = source.lower().strip()
allowed = "abcdefghijklmnopqrstuvwxyz1234567890"
slug = ""
while i < (len(source) - 0):
single_letter = source[i:i+1]
if single_letter in allowed:
slug += single_letter
elif not slug.endswith("-"):
#letter is not allowed
#check that the slug doesn't already end in a dash
slug += "-"
i = i + 1
return slugHere are short utility functions used to save to Google Cloud Storage in a Java App Engine application.
These snippets require the Java Cloud Storage library ( com.google.appengine.tools.cloudstorage.GcsService ). The bucket string is the name of the bucket, no gs:// required. folder_path is the subdirectory you want to save the file under. It should end with a forward slash. For example this/is/a/subdirectory/ . If you do not want a subdirectory, pass an empty string (“”). filename is the filename of the file you want to save, while contents is the data being saved.
There are two functions here. The top is intended to quickly save simple text/html/xml/etc files, while the bottom is intended to save binary files such as images, executables, and so forth. The bottom function includes a filetype argument, which should be filled with the appropriate MIME type. For example: image/png, image/jpg, application/octet-stream.
Reminder: GCS innately views files as individual objects. When you specify a subfolder, the filename for the object itself includes the folder path. When you read the file back from Google cloud storage, you need to include the folder path: for example this/is/a/subdirectory/example.png .
public static void saveStringToGCS(String bucket, String folder_path, String filename, String contents) throws IOException {
GcsService storage_service = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
GcsFilename to_file = new GcsFilename(bucket, folder_path + filename);
GcsFileOptions file_options = GcsFileOptions.getDefaultInstance();
storage_service.createOrReplace(to_file, file_options, ByteBuffer.wrap(contents.getBytes(StandardCharsets.UTF_8)));
}
public static void saveByteBufferToGCS(String bucket, String folder_path, String filename, ByteBuffer contents, String filetype) throws IOException {
GcsService storage_service = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
GcsFilename to_file = new GcsFilename(bucket, folder_path + filename);
GcsFileOptions file_options = (new GcsFileOptions.Builder()).mimeType(filetype).build();
storage_service.createOrReplace(to_file, file_options, contents);
}Here’s a quick example of using the Apache HTTPClient library to issue a simple GET request. The URL you’re connecting to is in endpoint, and the contents of the URL can be read from the BufferedReader response_reader.
HttpGet get_request = new HttpGet(endpoint);
get_request.addHeader(HttpHeaders.USER_AGENT, "Example Reader Service");
HttpResponse http_response = HttpClientBuilder.create().build().execute(get_request);
content_type_header = http_response.getFirstHeader("Content-Type").getValue();
int response_code = http_response.getStatusLine().getStatusCode();
if (response_code != 200) {
throw new IOException("Response code is not 200 OK. Response code: " + response_code + " : " + endpoint.toString());
}
//Pull out text response
InputStream response_is = http_response.getEntity().getContent();
BufferedReader response_reader = new BufferedReader(new InputStreamReader(response_is, "UTF-8"));The following code maps a bunch of unicode punctuation to their more normal ASCII counterparts. For example, the Unicode character LEFT DOUBLE QUOTATION MARK is mapped to the ASCII double quotation mark: “.
Hashtable<String, String> map = new Hashtable<String, String>();
map.put(new String(Character.toChars(0x00AB)), new String("\""));
map.put(new String(Character.toChars(0x00AD)), "-");
map.put(new String(Character.toChars(0x00B4)), "\'");
map.put(new String(Character.toChars(0x00BB)), "\"");
map.put(new String(Character.toChars(0x00F7)), "/");
map.put(new String(Character.toChars(0x01C0)), "|");
map.put(new String(Character.toChars(0x01C3)), "!");
map.put(new String(Character.toChars(0x02B9)), "\'");
map.put(new String(Character.toChars(0x02BA)), "\"");
map.put(new String(Character.toChars(0x02BC)), "\'");
map.put(new String(Character.toChars(0x02C4)), "^");
map.put(new String(Character.toChars(0x02C6)), "^");
map.put(new String(Character.toChars(0x02C8)), "\'");
map.put(new String(Character.toChars(0x02CB)), "`");
map.put(new String(Character.toChars(0x02CD)), "_");
map.put(new String(Character.toChars(0x02DC)), "~");
map.put(new String(Character.toChars(0x0300)), "`");
map.put(new String(Character.toChars(0x0301)), "\'");
map.put(new String(Character.toChars(0x0302)), "^");
map.put(new String(Character.toChars(0x0303)), "~");
map.put(new String(Character.toChars(0x030B)), "\"");
map.put(new String(Character.toChars(0x030E)), "\"");
map.put(new String(Character.toChars(0x0331)), "_");
map.put(new String(Character.toChars(0x0332)), "_");
map.put(new String(Character.toChars(0x0338)), "/");
map.put(new String(Character.toChars(0x0589)), ":");
map.put(new String(Character.toChars(0x05C0)), "|");
map.put(new String(Character.toChars(0x05C3)), ":");
map.put(new String(Character.toChars(0x066A)), "%");
map.put(new String(Character.toChars(0x066D)), "*");
map.put(new String(Character.toChars(0x200B)), " ");
map.put(new String(Character.toChars(0x2010)), "-");
map.put(new String(Character.toChars(0x2011)), "-");
map.put(new String(Character.toChars(0x2012)), "-");
map.put(new String(Character.toChars(0x2013)), "-");
map.put(new String(Character.toChars(0x2014)), "-");
map.put(new String(Character.toChars(0x2015)), "-");
map.put(new String(Character.toChars(0x2016)), "|");
map.put(new String(Character.toChars(0x2017)), "_");
map.put(new String(Character.toChars(0x2018)), "\'");
map.put(new String(Character.toChars(0x2019)), "\'");
map.put(new String(Character.toChars(0x201A)), ",");
map.put(new String(Character.toChars(0x201B)), "\'");
map.put(new String(Character.toChars(0x201C)), "\"");
map.put(new String(Character.toChars(0x201D)), "\"");
map.put(new String(Character.toChars(0x201E)), "\"");
map.put(new String(Character.toChars(0x201F)), "\"");
map.put(new String(Character.toChars(0x2032)), "\'");
map.put(new String(Character.toChars(0x2033)), "\"");
map.put(new String(Character.toChars(0x2034)), "\'");
map.put(new String(Character.toChars(0x2035)), "`");
map.put(new String(Character.toChars(0x2036)), "\"");
map.put(new String(Character.toChars(0x2037)), "\'");
map.put(new String(Character.toChars(0x2038)), "^");
map.put(new String(Character.toChars(0x2039)), "<");
map.put(new String(Character.toChars(0x203A)), ">");
map.put(new String(Character.toChars(0x203D)), "?");
map.put(new String(Character.toChars(0x2044)), "/");
map.put(new String(Character.toChars(0x204E)), "*");
map.put(new String(Character.toChars(0x2052)), "%");
map.put(new String(Character.toChars(0x2053)), "~");
map.put(new String(Character.toChars(0x2060)), " ");
map.put(new String(Character.toChars(0x20E5)), "\\");
map.put(new String(Character.toChars(0x2212)), "-");
map.put(new String(Character.toChars(0x2215)), "/");
map.put(new String(Character.toChars(0x2216)), "\\");
map.put(new String(Character.toChars(0x2217)), "*");
map.put(new String(Character.toChars(0x2223)), "|");
map.put(new String(Character.toChars(0x2236)), ":");
map.put(new String(Character.toChars(0x223C)), "~");
map.put(new String(Character.toChars(0x2264)), "<");
map.put(new String(Character.toChars(0x2265)), ">");
map.put(new String(Character.toChars(0x2266)), "<");
map.put(new String(Character.toChars(0x2267)), ">");
map.put(new String(Character.toChars(0x2303)), "^");
map.put(new String(Character.toChars(0x2329)), "<");
map.put(new String(Character.toChars(0x232A)), ">");
map.put(new String(Character.toChars(0x266F)), "#");
map.put(new String(Character.toChars(0x2731)), "*");
map.put(new String(Character.toChars(0x2758)), "|");
map.put(new String(Character.toChars(0x2762)), "!");
map.put(new String(Character.toChars(0x27E6)), "[");
map.put(new String(Character.toChars(0x27E8)), "<");
map.put(new String(Character.toChars(0x27E9)), ">");
map.put(new String(Character.toChars(0x2983)), "{");
map.put(new String(Character.toChars(0x2984)), "}");
map.put(new String(Character.toChars(0x3003)), "\"");
map.put(new String(Character.toChars(0x3008)), "<");
map.put(new String(Character.toChars(0x3009)), ">");
map.put(new String(Character.toChars(0x301B)), "]");
map.put(new String(Character.toChars(0x301C)), "~");
map.put(new String(Character.toChars(0x301D)), "\"");
map.put(new String(Character.toChars(0x301E)), "\"");
map.put(new String(Character.toChars(0xFEFF)), " ");
A slug in WordPress is the part of the URL that references the blog title. For example, if the URL looks like example.com/2019/this-is-my-test-post, then this-is-my-test-post would be a slug. Typically a slug is all letters and numbers, with any other character being replaced by a dash.
The below code fragment is a simple JS function to reduce a blog title down to a slug – you may want to add some additional code to guarantee a certain slug length.
var generateSlug = function generateSlug(title) {
var allowable_characters = "abcdefghijklmnopqrstuvwxyz1234567890";
//Builds our own slug
title = title.trim().toLowerCase();
var title_array = title.split("");
var outbound_title = " ";
for ( var i = 0; i < title_array.length; i++) {
var title_char = title_array[i];
if ( allowable_characters.indexOf(title_char) != -1 ) {
outbound_title = outbound_title + title_char;
}
else if ( outbound_title.split("").pop() != "-" ) {
outbound_title = outbound_title + "-";
}
}
console.log("Generated slug: " + outbound_title);
return outbound_title.trim();
}//end generateSlug
The following code example pulls out the last 5 published WordPress posts by their ID, extracts the first ID (which is the last published post ID) and then extracts the post’s tags into an array.
$return_tag_array is a list of the last published post’s tags.
//Get list of past published posts, up to 5
$args = array(
'numberposts' => 5,
'offset' => 0,
'category' => 0,
'orderby' => 'post_date',
'order' => 'DESC',
'include' => '',
'exclude' => '',
'meta_key' => '',
'meta_value' =>'',
'post_type' => 'post',
'post_status' => 'publish',
'suppress_filters' => true
);
$published_posts = wp_get_recent_posts( $args, ARRAY_A );
//Pull off the top most post, & retrieve the ID
$last_published_post = reset($published_posts);
$last_published_post_id = $last_published_post["ID"];
//TAGS
//Use the post ID to pull off tags
$tags = wp_get_post_tags($last_published_post_id);
$return_tag_array = array();
foreach ($tags as $tag) {
$tag_name = $tag->name;
array_push($return_tag_array, $tag_name);
}