Using Google SiteMaps

Today I wanted to create a dynamic google sitemap file for my site. So I started reading the documentation for creation of a sitemap. Google has a very good documented format, making all the points clear. you can vew it here. The Google Webmaster tools are also one of the powerful features where you could view the site stats, and thats where you need to submit your google sitemap.

The sitemaps protocol is defined at http://www.sitemaps.org/protocol.php

The sitemap is placed in the root directory which contains all the site urls for google to index pages. You can have a maximum of 50,000 urls per Sitemap. For additional urls, another sitemap file can be created and then linked with each other using SiteMap indexing file. Google does great at explaining all this stuff, so make sure you go through all the stuff in the link that I have put up here above to get the best of both worlds.

Generating the dynamic SiteMap file:

Generation is just like any other xml file, except that you have to adhere to the standards mentioned in the sitemap protocol.

Here's the code. You have to modify the query to suit your needs.

public partial class NewsSiteMap : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
DateTime now = DateTime.Now;
DataClassesDataContext db = new DataClassesDataContext();

var latestNewsArticles = (from articles in db.article_to_magazine_mappings
where
articles.magazine_id == 1 && articles.article.article_type_id != 20
select new
{
ID = articles.article.id,
Title = articles.article.web_title,
TitleLink = articles.TitleLink,
Author = articles.AuthorName,
TitleKeywords = articles.article.meta_tags,
Date = articles.publish_date,
LegacyID = articles.article.legacy_id
})
.Take(49999);


Response.Clear();

Response.ContentType = "text/xml";

using (XmlTextWriter writer = new XmlTextWriter(Response.OutputStream, Encoding.UTF8))
{
writer.WriteStartDocument();

writer.WriteStartElement("urlset");

writer.WriteAttributeString("xmlns", "http://www.sitemaps.org/schemas/sitemap/0.9");

foreach (var newsarticle in latestNewsArticles)
{
writer.WriteStartElement("url");

writer.WriteElementString("loc", ConcatenateUrls(newsarticle.TitleLink,newsarticle.Author,newsarticle.TitleKeywords,newsarticle.ID.ToString()));

writer.WriteElementString("lastmod", String.Format("{0:yyyy-MM-dd}", newsarticle.Date));

writer.WriteElementString("changefreq", "daily");

writer.WriteElementString("priority", "1.0");

writer.WriteEndElement();
}

writer.WriteEndDocument();

writer.Flush();

}

db.Dispose();
db = null;

Response.End();
}

protected string ConcatenateUrls(string title, string author, string keywords, string articleid)
{
return GetHost(title + "/By_" + author + keywords + "/" + articleid);
}
}

In the html, you just need to add the following line

<%@ Page Language="C#" AutoEventWireup="true" CodeFile="newsSitemap.aspx.cs" Inherits="NewsSiteMap" %>

Make sure you have encoded your content so that it doesnt break.
After you are done, submit your sitemap file to Google, and you are good!

Sort by Date for Google Site Search

Google Searches by Relevance by Default. One of the things that was on my TODO list was to have the articles search sorted by date, irrespective of the article relevance. So I had a radiobutton list which lets the users to choose if they want the results by relevance, or by date. There were not really lot of information about it on google blogs/forums, and nobody was sure whats the correct way to do it. After some clicks and head bashings, I found out the closest way to get to the results to sort by date is to have its own Annotation file and context file. So I logged into my google account and went to the custom search section, where I had customized my google site search. On the advanced Tab, I downloaded the Annotation file and the context file.

The Idea is that, when the user opts to sort by date, call these customized xml files to show results sorted out by date.

In the context file you will find the following tag

<Context>
    <BackgroundLabels>
      <Label name="_cse_bolrua_bzhw" mode="FILTER" />
      <Label name="_cse_exclude_bolrua_bzhw" mode="ELIMINATE" />
    </BackgroundLabels>
  </Context>

Add the following Label to the xml file and save it in your website directory.

<Label name="recent4" mode="BOOST"/>

This tag does the trick.

Also Just before the </GoogleCustomizations> tag, add the following line:

<Include type="Annotations" href="http://www.yourwebsite.com/annotations.xml"/>

This references the annotations file that you downloaded from the "Advanced Console"

There wont be any change in the annotations file, except that a copy will reside on the server. Make sure that when you change the site restrictions (Adding/Removing folders/files for site search), compare your downloaded version with the one on the Custom search.

Based upon the Radio Button selection, just select the annotation file:

if (rbSearchby.SelectedValue == "r") //Search by Relevance
            Response.Redirect(GetRoot() + "content/googlesearch.aspx?cx=01054907335329%3Abolrua_bzhw&cof=FORID%3A11&ie=UTF-8&q=" + HttpUtility.UrlEncode(SearchBox_TextBox.Text) + "&sa=Search");
        else
//Search by Date
            Response.Redirect(GetRoot() + "content/googlesearch.aspx?cref=http%3A%2F%2Fwww.yourwebsite.com%2Fgooglespecc.xml&cof=FORID%3A11&ie=UTF-8&q=" + HttpUtility.UrlEncode(SearchBox_TextBox.Text) + "&sa=Search");