SiteMap and Robots File

October 7, 2008 by Ari  
Filed under Search Engine Optimization

Sitemap and Robots

One of the best things you can do to get the search engines to notice you, is to install a sitemap and robots file on your blog or website.  Now I know that may sound a bit trivial for some and all, but it is really really simple.

First of all, you can get a sitemap very easily from XML-Sitemaps.com.  You simply follow four simple steps on their site and in no time at all, you will have the file you need for your site.  Very simply put: upload that file to your root directory (the place that is the bottom most tier of your website – example: c:/ would be the root of a PC, and so the same would apply to a website or blog, only the root is where the website is being hosted).

Once you have done so, it is a great idea to sign up with Google Webmaster Tools so you can tell Google exactly what you want crawled on your site.  You will be able to designate what to crawl and what not to.  Although some crawling robots out there, ignore the request and crawl everything anyway (usually these are spam search robots). 

SiteMap XML

A sitemap basically tells Google and other search engine crawling robots, what your site looks like in terms of a map.  It says to the crawling spiders what is okay to index and what is not.  It is important to have a sitemap on your site, because even though the internet looks for content all the time, they won’t always know you are there, unless you make an effort to tell them.

Along with a sitemap, it is a good idea to install a robots.txt file so that it can work in conjunction with the sitemap in that it will tell any other crawling search "spider", what to search and what to leave alone.  And although a sitemap and robots file sound the same, they are not.  The sitemap tells the search engines what the landscape looks, like and the robots file tells them what to index and crawl, and what to leave alone.

Robots.txt

You can easily get a robots.txt file the same way you got a sitemap from XML-Sitemaps.  Just simply upload this .txt file to your root directory also and you are set to go.  If you want to find a sample robots.txt file to upload to you site, just search robots.txt on the net and you will see a sample come up.  Save it as a txt file and upload it to the root directory.

The internet is so much friendlier to your site or blog when you have these things in place.  You can visit Google Webmaster Tools for more information or XML-Sitemaps too. 

  • Share/Bookmark
Press Ctrl+D to Bookmark this Site: google webmaster tools  search engines  simple steps  XML-Sitemaps  other search engine  website  search robots  root directory

GoingUp.com is a Complete Solution

I don’t know how detailed you are about your website or its optimization needs, but there are tools out there for whatever the need may be.  In this case, I am referring to GoingUp.com.  GoingUp.com brings a unique twist to website optimization by putting there money where their mouth is when they talk about HTML validation.  You see, a lot of SEO recommendations say "make sure your website is XHTML compliant" but they don’t tell you where you’re sight has issues.  The GoingUp.com/Analyzer does!  It gives you a complete and specific line by line play of every line of code in your website that needs attention.  Using this method, you can literally make all your XHTML errors disappear if you are at least basically familiar with code (although I personally think this area is really for intermediates and above). 

Nonetheless, if you ever wondered if the way you entered a line of code works or not, this analyzer will tell you.  When I ran it on my site I found over 100 errors that I had not even considered like abandoned/open/orphaned tags.  Unless you do this with a regular editing program like HTML Toolkit or something similar, you can find yourself missing some of these once in a while, especially if you are a novice at code and are using programs or plug-ins to help your site execute what you want it to do.  There tools are powerful.

Website tools have come a long way as search engines continuously change their search algorithms.  But companies like GoingUp.com are leaving companies of days old in the dust.  They have tag generators and SEO optimizers that really get down to the details way beyond anything I’ve ever seen before.  And these guys even offer a free plug-in of their own that will install into your WordPress, Blogger, or Joomla folders, for on the fly statistics. 

I highly recommend that you give GoingUp.com a try and a visit.  I’m still in wow.

going up dot com

  • Share/Bookmark
Press Ctrl+D to Bookmark this Site: complete solution  toolkit  website tools  SEO  method  program  statistics  folders  xhtml  optimizers  WordPress  Search  optimization  Solution

The 3 Pillars on Internet Ambitions Part 2 of 4

< PREVIOUS PART HERE

 

 

Pillar Two: Content

How many times are you going to read somewhere that “content is king?”  Everyone says it is in some form or another and goes further by giving you’re their perspective, but the bottom line is: where do you get it? And, how much of it do you need?

    Sourcing:  The internet is a place that is constantly hungry for information.  And
    unfortunately, the majority of the people that go to the internet are looking for
    information and only the minority of the people are providing that information.
    Therefore, you can almost guarantee yourself a spot on the information highway. 
    Sourcing begins with you.  Tell the world what you are good at.  And don’t try to be the
    expert.  Just share your opinions at a minimum, and make sure to communicate those
    thoughts and ideas -regularly.
   
    Volume:     Let me put it this way, when you setup your domain you created your first index
    (i.e., www.yoursite.com).  Google will index you eventually and so will MSN and Yahoo. 
    However, for most bloggers’ efforts to be considered even mildly successful, they have to
    reach a minimum level of at least 500 indexes going to your domain before you start to
    see viable quantities of traffic on your site.  By indexes I don’t mean inbound links.
    And this “indexing” happens only when you post to your blog.  More posts, more indexing.
    Additionally, if you assign tags (keywords) to your posts, you may increase the index. 
    In the above example, if you have written 100 posts, then you will have 100 indexes, i.e:

             Site Index URL Number 1:       www.yoursite.com
             Site Index URL Number 100:   www.yoursite.com/post-name-etc/

   Now add “simple tags” and “sitemap” your website, and you have effectively raised your
   indexes (URLs) to over 200 by now.

             Site Domain Index: 1
             Site Post Indexes: 100
             Site Tag Indexes: 100

   Congratulations, you now have 201 indexes (assuming the search engines index your total
   URLs) that reference your presence when people search (regardless of whether it’s a
   social network or a search engine).
   
   Visit Google.com and in the search bar type site:yourdomain.com to see your indexes. 
   Where ‘yourdomain’ is replaced by your domain or the website of your choice.

     Links:  All conversations about search engine optimization (SEO) and social media
     optimization (SMO) end with inbound links.  Google, MSN and Yahoo use them to
     determine your site’s “rank” in their search results.  The more websites that link to you,
     the higher your score and authority.  Obviously if you are writing content and enhancing
     the information highway, then you don’t have to worry about links, they will eventually
     get around to you.

CONTINUED HERE <

  • Share/Bookmark
Press Ctrl+D to Bookmark this Site: pillar  blog  engine  website  index url  traffic  indexes  viable quantities  bottom line  inbound links  optimization  Content  google  yahoo