When an application makes an URLFetch request, App Engine adds the following text as the User-Agentheader:
AppEngine-Google; (+http://code.google.com/appengine;
appid: YOUR_APPLICATION_ID_HERE)
Even if the application sets a custom user agent header, App Engine will append the above text to the header.
This can be annoying because there are some servers and services that rate limit based on the user agent. If there is a human reviewing the request logs, it can be confusing to see a stream of largely-identical user agent strings.
It’s good practice to set a descriptive user agent for all URL fetches. It’s even better if you can write your user agent with App Engine’s required text in mind. For instance, consider writing user agent headers like this one: App Engine Example Crawler hosted by. When App Engine appends its required text to the end of this, the receiving server will see an user agent of:
App Engine Example Crawler hosted by
AppEngine-Google; (+http://code.google.com/appengine;
appid: YOUR_APPLICATION_ID_HERE)
This user agent header looks cleaner, neater, and is easier for a human to understand.
Here is the above in code form:
String user_agent = "App Engine Example Crawler";
user_agent += " hosted by ";//After this, GAE will append the identifier.
connection.setRequestProperty("User-Agent", user_agent);
The connection variable represents a java.net.HttpURLConnection object.