This Google Cloud post came up on my LinkedIn feed, and I thought it deserved a special mention:
Google Cloud LinkedIn Post
Missing Logs – Google Logs Viewer
I opened a new GCP project to host a Python application when I hit a problem – my logging.info() and logging.warn() statements weren’t showing up in my logs. Then I realized the standard error and standard out streams weren’t selected in logging!
If you’re missing log information, make sure to select the correct streams in the second dropdown box, as in below:
PhantomJSCloud Error: Invalid Character ‘u’ – Python
I use PhantomJSCloud to take screenshots of web apps. While writing new code, I noticed this error coming back from the server:
{
"name":"HttpStatusCodeException",
"statusCode":400,
"message":"Error extracting userRequest. innerException: JSON5: invalid character 'u' at 1:1",
"stack":[
"no stack unless env is DEV or TEST, or logLevel is DEBUG or TRACE"
]
}The problem came because the request wasn’t JSON-encoding the object; the fix looks like this (using the requests library):
post_data_string = json.dumps(python_object_here, sort_keys=True, indent=3, separators=(',', ': '))
r = requests.post('https://phantomjscloud.com/api/browser/v2/', data=post_data_string)Turning A Date Into A Folder Path – Node
This short code segment turns today’s date into a folder path. For example, today’s date of April 2, 2019 would be turned into 2019/04/02. I use it for categorizing date related documents into appropriate folders within GCS.
var datestamp_obj = new Date( (new Date()).getTime() + (-6) * 3600 * 1000 );
var datestamp_slash = datestamp_obj.getFullYear() + "/";
datestamp_slash += datestamp_obj.getMonth().toString().padStart(2, "0") + "/" + datestamp_obj.getDate().toString().padStart(2, "0");
Using Google Protobuf Timestamp In PHP
This is more of a documentary post because I haven’t seen documentation on Google’s Timestamp class anywhere.
Google’s libraries – in particular, the GCP libraries for datastore/tasks/etc – use the Google/Protobuf/Timestamp class to represent time. Timestamp is a simple wrapper around the number of seconds since UNIX epoch, in the UTC timezone. For example here is how to create a Timestamp reflecting the current date and time, plus 2 minutes into the future (120 seconds):
use Google\Protobuf\Timestamp;
$future_time_seconds = time() + 120;
$future_timestamp = new Timestamp();
$future_timestamp->setSeconds($future_time_seconds);
$future_timestamp->setNanos(0);There are equivalent classes and functions for Python/Java/Go/other languages that Google Cloud supports.
Using the Timestamp class – especially setting up future dates – is necessary for configuring my favorite Google Cloud service: Cloud Tasks. A Cloud Task can accept a future date to run at, thereby giving you a way to queue up and delay execution of an activity. For example, see the below screenshot: I’ve created 3 tasks 20 seconds ago, yet they’re set for a future execution 3 minutes 45 seconds from now:
Creating A WordPress Blog Slug Part 2 – Python
In a previous post, I posted a sample NodeJS function to assemble a WordPress blog slug. I ended up rewriting part of the larger application (and the function itself) in Python.
In the below function, source is a blog title string, and it returns a slug suitable for use in a blog URL.
def generate_slug(source):
i = 0
source = source.lower().strip()
allowed = "abcdefghijklmnopqrstuvwxyz1234567890"
slug = ""
while i < (len(source) - 0):
single_letter = source[i:i+1]
if single_letter in allowed:
slug += single_letter
elif not slug.endswith("-"):
#letter is not allowed
#check that the slug doesn't already end in a dash
slug += "-"
i = i + 1
return slugUtility Functions To Save To Google Cloud Storage
Here are short utility functions used to save to Google Cloud Storage in a Java App Engine application.
These snippets require the Java Cloud Storage library ( com.google.appengine.tools.cloudstorage.GcsService ). The bucket string is the name of the bucket, no gs:// required. folder_path is the subdirectory you want to save the file under. It should end with a forward slash. For example this/is/a/subdirectory/ . If you do not want a subdirectory, pass an empty string (“”). filename is the filename of the file you want to save, while contents is the data being saved.
There are two functions here. The top is intended to quickly save simple text/html/xml/etc files, while the bottom is intended to save binary files such as images, executables, and so forth. The bottom function includes a filetype argument, which should be filled with the appropriate MIME type. For example: image/png, image/jpg, application/octet-stream.
Reminder: GCS innately views files as individual objects. When you specify a subfolder, the filename for the object itself includes the folder path. When you read the file back from Google cloud storage, you need to include the folder path: for example this/is/a/subdirectory/example.png .
public static void saveStringToGCS(String bucket, String folder_path, String filename, String contents) throws IOException {
GcsService storage_service = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
GcsFilename to_file = new GcsFilename(bucket, folder_path + filename);
GcsFileOptions file_options = GcsFileOptions.getDefaultInstance();
storage_service.createOrReplace(to_file, file_options, ByteBuffer.wrap(contents.getBytes(StandardCharsets.UTF_8)));
}
public static void saveByteBufferToGCS(String bucket, String folder_path, String filename, ByteBuffer contents, String filetype) throws IOException {
GcsService storage_service = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
GcsFilename to_file = new GcsFilename(bucket, folder_path + filename);
GcsFileOptions file_options = (new GcsFileOptions.Builder()).mimeType(filetype).build();
storage_service.createOrReplace(to_file, file_options, contents);
}A HTTP GET Example Using The Apache HTTPClient Library
Here’s a quick example of using the Apache HTTPClient library to issue a simple GET request. The URL you’re connecting to is in endpoint, and the contents of the URL can be read from the BufferedReader response_reader.
HttpGet get_request = new HttpGet(endpoint);
get_request.addHeader(HttpHeaders.USER_AGENT, "Example Reader Service");
HttpResponse http_response = HttpClientBuilder.create().build().execute(get_request);
content_type_header = http_response.getFirstHeader("Content-Type").getValue();
int response_code = http_response.getStatusLine().getStatusCode();
if (response_code != 200) {
throw new IOException("Response code is not 200 OK. Response code: " + response_code + " : " + endpoint.toString());
}
//Pull out text response
InputStream response_is = http_response.getEntity().getContent();
BufferedReader response_reader = new BufferedReader(new InputStreamReader(response_is, "UTF-8"));Mapping Unicode Punctuation To ASCII Punctuation
The following code maps a bunch of unicode punctuation to their more normal ASCII counterparts. For example, the Unicode character LEFT DOUBLE QUOTATION MARK is mapped to the ASCII double quotation mark: “.
Hashtable<String, String> map = new Hashtable<String, String>();
map.put(new String(Character.toChars(0x00AB)), new String("\""));
map.put(new String(Character.toChars(0x00AD)), "-");
map.put(new String(Character.toChars(0x00B4)), "\'");
map.put(new String(Character.toChars(0x00BB)), "\"");
map.put(new String(Character.toChars(0x00F7)), "/");
map.put(new String(Character.toChars(0x01C0)), "|");
map.put(new String(Character.toChars(0x01C3)), "!");
map.put(new String(Character.toChars(0x02B9)), "\'");
map.put(new String(Character.toChars(0x02BA)), "\"");
map.put(new String(Character.toChars(0x02BC)), "\'");
map.put(new String(Character.toChars(0x02C4)), "^");
map.put(new String(Character.toChars(0x02C6)), "^");
map.put(new String(Character.toChars(0x02C8)), "\'");
map.put(new String(Character.toChars(0x02CB)), "`");
map.put(new String(Character.toChars(0x02CD)), "_");
map.put(new String(Character.toChars(0x02DC)), "~");
map.put(new String(Character.toChars(0x0300)), "`");
map.put(new String(Character.toChars(0x0301)), "\'");
map.put(new String(Character.toChars(0x0302)), "^");
map.put(new String(Character.toChars(0x0303)), "~");
map.put(new String(Character.toChars(0x030B)), "\"");
map.put(new String(Character.toChars(0x030E)), "\"");
map.put(new String(Character.toChars(0x0331)), "_");
map.put(new String(Character.toChars(0x0332)), "_");
map.put(new String(Character.toChars(0x0338)), "/");
map.put(new String(Character.toChars(0x0589)), ":");
map.put(new String(Character.toChars(0x05C0)), "|");
map.put(new String(Character.toChars(0x05C3)), ":");
map.put(new String(Character.toChars(0x066A)), "%");
map.put(new String(Character.toChars(0x066D)), "*");
map.put(new String(Character.toChars(0x200B)), " ");
map.put(new String(Character.toChars(0x2010)), "-");
map.put(new String(Character.toChars(0x2011)), "-");
map.put(new String(Character.toChars(0x2012)), "-");
map.put(new String(Character.toChars(0x2013)), "-");
map.put(new String(Character.toChars(0x2014)), "-");
map.put(new String(Character.toChars(0x2015)), "-");
map.put(new String(Character.toChars(0x2016)), "|");
map.put(new String(Character.toChars(0x2017)), "_");
map.put(new String(Character.toChars(0x2018)), "\'");
map.put(new String(Character.toChars(0x2019)), "\'");
map.put(new String(Character.toChars(0x201A)), ",");
map.put(new String(Character.toChars(0x201B)), "\'");
map.put(new String(Character.toChars(0x201C)), "\"");
map.put(new String(Character.toChars(0x201D)), "\"");
map.put(new String(Character.toChars(0x201E)), "\"");
map.put(new String(Character.toChars(0x201F)), "\"");
map.put(new String(Character.toChars(0x2032)), "\'");
map.put(new String(Character.toChars(0x2033)), "\"");
map.put(new String(Character.toChars(0x2034)), "\'");
map.put(new String(Character.toChars(0x2035)), "`");
map.put(new String(Character.toChars(0x2036)), "\"");
map.put(new String(Character.toChars(0x2037)), "\'");
map.put(new String(Character.toChars(0x2038)), "^");
map.put(new String(Character.toChars(0x2039)), "<");
map.put(new String(Character.toChars(0x203A)), ">");
map.put(new String(Character.toChars(0x203D)), "?");
map.put(new String(Character.toChars(0x2044)), "/");
map.put(new String(Character.toChars(0x204E)), "*");
map.put(new String(Character.toChars(0x2052)), "%");
map.put(new String(Character.toChars(0x2053)), "~");
map.put(new String(Character.toChars(0x2060)), " ");
map.put(new String(Character.toChars(0x20E5)), "\\");
map.put(new String(Character.toChars(0x2212)), "-");
map.put(new String(Character.toChars(0x2215)), "/");
map.put(new String(Character.toChars(0x2216)), "\\");
map.put(new String(Character.toChars(0x2217)), "*");
map.put(new String(Character.toChars(0x2223)), "|");
map.put(new String(Character.toChars(0x2236)), ":");
map.put(new String(Character.toChars(0x223C)), "~");
map.put(new String(Character.toChars(0x2264)), "<");
map.put(new String(Character.toChars(0x2265)), ">");
map.put(new String(Character.toChars(0x2266)), "<");
map.put(new String(Character.toChars(0x2267)), ">");
map.put(new String(Character.toChars(0x2303)), "^");
map.put(new String(Character.toChars(0x2329)), "<");
map.put(new String(Character.toChars(0x232A)), ">");
map.put(new String(Character.toChars(0x266F)), "#");
map.put(new String(Character.toChars(0x2731)), "*");
map.put(new String(Character.toChars(0x2758)), "|");
map.put(new String(Character.toChars(0x2762)), "!");
map.put(new String(Character.toChars(0x27E6)), "[");
map.put(new String(Character.toChars(0x27E8)), "<");
map.put(new String(Character.toChars(0x27E9)), ">");
map.put(new String(Character.toChars(0x2983)), "{");
map.put(new String(Character.toChars(0x2984)), "}");
map.put(new String(Character.toChars(0x3003)), "\"");
map.put(new String(Character.toChars(0x3008)), "<");
map.put(new String(Character.toChars(0x3009)), ">");
map.put(new String(Character.toChars(0x301B)), "]");
map.put(new String(Character.toChars(0x301C)), "~");
map.put(new String(Character.toChars(0x301D)), "\"");
map.put(new String(Character.toChars(0x301E)), "\"");
map.put(new String(Character.toChars(0xFEFF)), " ");
Creating A WordPress Blog Slug – NodeJS
A slug in WordPress is the part of the URL that references the blog title. For example, if the URL looks like example.com/2019/this-is-my-test-post, then this-is-my-test-post would be a slug. Typically a slug is all letters and numbers, with any other character being replaced by a dash.
The below code fragment is a simple JS function to reduce a blog title down to a slug – you may want to add some additional code to guarantee a certain slug length.
var generateSlug = function generateSlug(title) {
var allowable_characters = "abcdefghijklmnopqrstuvwxyz1234567890";
//Builds our own slug
title = title.trim().toLowerCase();
var title_array = title.split("");
var outbound_title = " ";
for ( var i = 0; i < title_array.length; i++) {
var title_char = title_array[i];
if ( allowable_characters.indexOf(title_char) != -1 ) {
outbound_title = outbound_title + title_char;
}
else if ( outbound_title.split("").pop() != "-" ) {
outbound_title = outbound_title + "-";
}
}
console.log("Generated slug: " + outbound_title);
return outbound_title.trim();
}//end generateSlug