Translation from English into English isn’t supported

• <meta name="robots" content="noarchive" /> seems to be the “standard” one, “officially supported by Google, Yahoo!, Bing and Ask” (source). This tag is enough, the other methods I’ll list are just here for informational purpose. On a side note, this noarchive option seems pretty old, I found a reference to it back in 2000.
• <meta name="googlebot" content="noarchive" /> is Google-specific, that’s if you want to disable caching just for Google.
• <meta name="robots" content="nocache " /> is Bing-specific. But since Bing supports noarchive too, there’s no reason to use it…

On a side note, it’s also possible to disable the page snippet/abstract, at least on Google, using the following: <meta name="googlebot" content="nosnippet" />. Note that this option will also disable the cache.

Sources:

• Disabling the Google Cache:New Meta Tag : NOARCHIVE
• 6 methods to control what and how your content appears in search engines
• Official Google Blog: The Robots Exclusion Protocol
• Special Google searches – Webmaster Tools Help
• Meta tags – Webmaster Tools Help

As promised at the beginning, the uninteresting complete history:
- someone sent me the following funky Google error message:

In class com.google.gaia.frontend.page.ServiceLogin: com.google.gaia.client.GaiaRemoteException: Backend not available (backend says: Cookie decode task expired while queued)

- I search for it, then for parts of it, and eventually ended up on Google Blogoscoped
- this site referred at some point to a Wikipedia page dedicated to criticism of Google, where I found out that caching can be disabled.

User-agent: MJ12bot
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: twiceler
Disallow: /
User-agent: Yandex
Disallow: /
User-agent: Baiduspider
Disallow: /
User-agent: spbot
Disallow: /
User-agent: ScoutJet
Disallow: /
User-agent: Linguee
Disallow: /

• MJ12bot is the bot used by Majestic-12. They seem to have an index as big as Google’s, with over 1 trillion pages (source), but they currently focus on their SEO tools and the search engine itself doesn’t really give good results yet. They’re planning to work on it a lot this year, though (source).
• dotbot is the bot of dotnetdotcom.org. I don’t really have a clue of what they’re trying to achieve. They’re not a search engine (yet?) and all they do is offer to download as a huge archive their whole index. The latest version to date was released 10 months ago, in April 2009, and contains only 600k pages according to the description… Not worth checking out ATM IMO.
• twiceler is the bot of Cuil. They say their index is 127 billion pages big, which is not bad for a search engine I had never heard about. They let you choose your preferred language (8 languages available) and the layout is rather pretty. Too bad they can only display 10 results per page, plus the results aren’t always very properly sorted. But this means that they will sometimes give you interesting results that even Google didn’t manage to find! So, definitely worth trying.
• Yandex is from Yandex.ru, a Russian search engine. No figures about their index, they seem to give good results although not extensive.
• Baiduspider is from Baidu, a Chinese search engine. For some reason a Google search about it gives a lot of results about people having troubles blocking it. The results don’t seem very good, at least patheticcockroach.com and wiki4games.com don’t show up when I look for them…
• spbot is from SEOprofiler by Axandra GmbHSE. So apparently only a SEO site, not a search engine.
• ScoutJet is the bot of Blekko, a yet-to-be-released search engine.
• Linguee describes itself as a “German-English dictionary”. So I’m not sure why they have a crawler…

On a side note, while searching for the above bots, I also found Slurp, which is Yahoo’s crawler, and Teoma, which is ask.com’s crawler.

MediaWiki does have a function to generate its own sitemap. The script is located in the maintenance folder and is called generateSitemap.php. It have to be run from the console (thus you need the php-cli package). In order to have a regularly updated map, we chose to configure its generation to run as a daily Cron job. The command is quite straightforward:

php /home/username/www/maintenance/generateSitemap.php --fspath="/home/username/www" --server="http://www.wiki4games.com" --compress=no

NB: as for some or most of the maintenance scripts, this script requires AdminSettings.php (in the wiki’s root folder) to be filled properly. This file basically contains the login and password for a database user, and can be created based on AdminSettings.sample.

• /home/username/www/maintenance/generateSitemap.php is obviously the absolute path to generateSitemap.php
• –fspath=”/home/username/www” is the path to the folder where you want to save the sitemaps. The script will generate quite a few files, at it will create one sitemap per namespace. Yet, if you want to submit it to Google, you must not place the sitemaps in a subfolder of the wiki (see farther)
• –server=”http://www.wiki4games.com” is optional (use it if auto detection fails… or systematically if you prefer)
• –compress=no disable sitemap compression. I don’t know if Google is able to read compressed sitemaps, so I disabled it (default is yes)

Submitting the map

To submit the sitemap to Google, go verify your site to Google Webmaster Tools (if you have soft 404 errors you’ll need to choose the meta tag verification, to add a meta tag just edit your default skin – eg skins/MonoBook.php -, search for “<head>” and add the meta tag somewhere after it) then submit the sitemap index, which is named something like sitemap-index-[database name]-[table prefix].xml. And wait for it to be crawled (should take a few minutes).

Fixing the errors

URL not allowed

URL not allowed
This url is not allowed for a Sitemap at this location.

This means that you placed your sitemap either in a level inferior to the pages you are listing (ie in a subfolder), either on another domain (if you’re listing https://www.mywiki.com, you can’t place the sitemaps in http://www.mywiki.com nor in https://www.mywiki.com/mymaps). So just move the sitemap to an appropriate place (very same domain and higher or same level as all the URLs listed).

Invalid URL

Invalid URL
We’ve detected that a Sitemap you’ve listed doesn’t include the full URL.

This is just a warning that shouldn’t prevent indexing. Yet for a nicer icon (valid instead of warning ) you may want to fix it. This warning is due to the fact that the sitemap index created by the script uses relatives URLs instead of absolute ones. To fix it, you’ll have to edit the script:

• open generateSitemap.php
• search for function indexEntry
• in this function, find "\t\t$filename\n" and add the path need to make the URL complete. For instance if your sitemap index is on http://www.mywiki.com/sitemap-index-wiki4games-wg_.xml, replace this text with "\t\thttp://www.mywiki.com/$filename\n" (note the slash!)

Regenerate your sitemap, then resubmit it to Google. All should be fine now… except if one of your maps has over 50k URLs (Google doesn’t accept this, you need to split them… and I don’t know how to do this :/)

=> First summary: I got an ad e-mail saying I’d get 100€ of free advertising without anything to pay (the e-mail clearly stated that the 5€ activation fee could be deducted from the 100€ offered), in reality I got 110€ of advertising credits but paid them 10€

Here goes the campaign, not too hard to set up (the “easy” mode is what made me waste a lot of time at the beginning, and now that I switched to the “Standard Edition” – ie the edition not for retards – I have a nice undismissable sticky polluting my Account Snapshop page – see screenshot), and running smooth at start: over 50,000 views within the first 12 hours.

Something that struck me, though, is the very high default CPC: if you use the noob-mode campaign set up, it’s around 0.69€ (which, as of today, is pretty close to 1$). But even though I lowered it to 0.10€, the ads displayed fast… at first…

I also chose some keywords to display the ads on some relevant Google Search result pages. Wiki4Games being a video game wiki, I notably chose “video game wiki” and “video games wiki” as keywords. Our current position to these keywords is around 4-5 (see screenshot), over more than 22 millions other results.

Adwords gently complained that my max CPC bid was a bit too low which would result in a bad placement and fewer impressions, still I got over 75,000 during the first day, then 50,000, then a bit less, and then it fell to almost nothing – less than 900 today (with no impression on Google Search, only impressions on the Adsense network). And the worst part is, Adwords claim my keywords are irrelevant. More precisely, they became irrelevant: during the first days, my keywords were rated 7/10 (one of them was even 10/10). Since 2 days, they are rated 2/10, that’s right, my website is in the top 5 over 22,000,000 results for this keyword yet the keyword is irrelevant!

So is Google irrelevant or am I too cheap for them? I guess I’ll have to increase my CPC bids and see if my relevance increases too… I’ll then see if Google violates the very web neutrality they made so much ado about, or if Adwords just sucks at keywords…

PS: and I forgot to mention, my second ad (pointing to a page about Dune) has been pending for review for 5 days, and counting…

Update on July 31: Dune ad still pending review! (a month and a half after submission!)
Update on Aug 29: The Dune ad was finally approved, at some unknown point in August

Due to FeedBurner’s merging into Google, we will no longer use their service to mirror our feed. Thus we encourage you to change your subscription from FeedBurner to our Atom feed here.

Say thank you for (trying to) save you from yet another Google monopoly

XYZ is offline. Messages you send will be delivered when XYZ comes online.

Thus an instant message is delivered as soon as your friend is online. Which means that if he already is, he’ll get your IM right away. Even better, the IM will pop in as if he was normally connected. I.e. with the little “gloong” (or whatever you may call it) noise if the Gmail window is in the background – and provided he set the sounds on, of course.
And the very best, if you friend just starts typing, you’ll get notified by the usual “[XYZ] is typing…”.

So, enjoy Gtalking your “offline” friends
And when you want to stay really invisible, don’t place your cursor in the reply box…

Read more | Digg it

Documents seized during raids on the homes of insurgents last week uncovered print-outs from photographs taken from Google.

Read on