For many e-commerce store owners using CRE Loaded SEO URL module, the challenge has been how to effectively utilize CRE Loaded SEO URL module with product search engine and xml sitemap feeds like Google Product Search and Google XML Sitemap feeds . I have been able to get CRE Loaded SEO URL module to work with both (Google Product Search and Google XML Sitemap feeds); the same method can be used to generate feeds for Yahoo, MSN, Edgeio(Google), Mysimon and other product search engine and xml sitemaps.

Below is a listing of the top 10 search engines in the United States. These listings are displayed in ranking order for the top 5 search engines and alphabetical order for 2nd tier search engines along with links to their FAQ’s and Add/Submit URI pages. (source: http://www.seoconsultants.com)

Search Engine Statistics - What Market Share?

2007 December: According to 2007 December figures from Hitwise…

1. Google has the largest market share of U.S. based web searches at 65.98%
   • 2007 November 65.10%
   • 2007 October 64.49%
   • 2007 September 63.55%
   • 2006 October 60.94%
2. Yahoo is second with a market share of 20.88%
   • 2007 November 21.21%
   • 2007 October 21.65%
   • 2007 September 22.55%
   • 2006 October 22.34%
3. MSN is third with a market share of 7.04%
   • 2007 November 7.09%
   • 2007 October 7.42%
   • 2007 September 7.83%
   • 2006 October 10.72%
4. Ask is fourth with a market share of 4.14%
   • 2007 November 4.63%
   • 2007 October 4.76%
   • 2007 September 4.32%
   • 2006 October 4.34%

“Note: Data is based on four week rolling periods (ending Oct. 27, 2007, Sept. 29, 2007, Oct. 28, 2006) from the Hitwise sample of 10 million US Internet users.”

Source: Hitwise - Dominance continues with a six percent increase over last year.

Top 5 Search Engines in the United States (The Majors) by Rank

1. Google
   Free Submit: http://www.google.com/addurl/
2. Yahoo! Search
   Free Submit: http://submit.search.yahoo.com/free/request/
3. MSN Live Search
   Free Submit: http://search.msn.com/docs/submit.aspx
4. AOL Search - Powered by Google
   Submit: https://adwords.google.com/select/
5. Ask - Powered by Teoma
   FAQs: http://static.wc.ask.com/en/docs/about/webmasters.shtml#18

Alphabetical Listing of 2nd Tier Search Engines

1. AltaVista - Powered by Yahoo!
   Free Submit: http://submit.search.yahoo.com/free/request/
2. Fast (AlltheWeb.com) - Powered by Yahoo!
   Submit: http://submit.search.yahoo.com/free/request/
3. Gigablast - Real time spidering, really!
   Free Submit: http://www.gigablast.com/addurl
4. Netscape Search - Powered by Google
   Free Submit: http://www.google.com/addurl/
5. Snap.com - Portions powered by: Gigablast, Smarter.com, SimplyHired.com, X1 Technologies, Inc. and Enhanced by Ask.com
   Submit: http://www.snap.com/

Here are the top search engines you submit your site for free.

Google: http://www.google.com/addurl/?continue=/addurlWindowsLive/MSN: http://search.msn.com/docs/submit.aspx

Open Directory Project: http://www.dmoz.org/add.html

Librarian’s Internet Index: http://lii.org/cs/lii/create/todo

Yahoo Search: https://siteexplorer.search.yahoo.com/submit

Alexa: http://www.alexa.com/site/help/webmasters

These tools/software are very important for any on-line stores:

Goolge Analytics (http://www.google.com/analytics/)
Goolge Webmastertool (http://www.google.com/webmasters/tools/)
Google Product Search (http://base.google.com/)
Add and submit Site Map
Add SEO url
Submit url

• Google -http://www.google.com/addurl/
• Yahoo - https://siteexplorer.search.yahoo.com/submit
• MSN Live Search - http://search.msn.com.sg/docs/submit.aspx

They should be implemented as soon as feasible.

I get questions on why Google Webmaster Tool shows numerous links as restricted. The reason for this may due to information in your robots.txt file and/or information in your sitemap. If you have both, it’s a good idea to check the restricted links against robots.txt and sitemap file. Overall the best way to troubleshoot this problem is to follow Google’s guideline:

Debugging blocked URLs

From my experience many of these restricted links are valid; good examples are buy now and product review links. It’s obvious that a search engine robot will neither be able to make a purchase or write a product review. For the case of buy now; in product_listing.php, you can wrap the code in an if statement like so:

case 'PRODUCT_LIST_BUY_NOW':
$lc_align = 'center';
if($session_started) $lc_text = ‘…’;
else $lc_text = ”;
break;

While looking for information on how to submit sitemap to MSN, I can across this piece of information from Yahoo blog. To add sitemap to robots.txt, just add the following line:

Sitemap: http://www.example.com/sitemap.xml

In case you have multiple Sitemaps, you can point to your sitemap index
Sitemap: http://www.example.com/sitemapindex.xml

To do this, simply add the following line to your robots.txt file:

Sitemap: http://www.example.com/sitemap.xml

Please provide the complete URL for your Sitemap on this line. We will pick it up wherever you put it in your robots.txt file. This directive is not specific to user-agent. If you have multiple Sitemaps, you can point to your Sitemap index file on this line.

It is good to know that one does need to manually submit site map to Google, Yahoo and MSN. I hope this is going to be the standard very soon.