Do manual web crawling and look up site maps

Any one using a search engine is aware of crawlers that move across the internet finding us relevant links trying to go along with the relevancy of search key word that you enter. One of the main ideas of Search crawlers is to look up relevancy from one site to another and using Page Ranks its easier for it to find relevant information. A web crawler decides that the higher the page rank the more relevant is the information.

Google has tried allowing use to crawl the web for results manually byintroducing Wonder Wheel and Google Swirl for image search. These tools are high on eye candy but I think need to improve on relevancy of results. When I first used walk2web I was impressed with it’s presentation style but did not really use it much for getting to know how useful it can be.

About Walk2Web

• Walk2Web allows you to enter a URL of a website you want to navigate or look deeper into and has a cool interface to show important links.
• It usually shows to the right hand corner links with some basic information. You can click on the link and takes your to the blog article directly.
• I gives you something of a site map of the URL you are  digging deeper into. With internal links and other links you can get start exploring from one link to another which basically resembles manual crawling. While you click individual link it also produces a screen-shot to the right hand side.
• This tool can be quite useful to get a visual simulation of how your website will be looked up or deconstructed by a search engine.
• Apart from this I think its a must use for blog owners. As often blogs keep growing and your website might not well organized. How well organized or unorganized your blog site is can be easily found out by checking out how you can make your way around on the website.

Take the example of Devils Workshop. If you enter the keywords ‘Join Devils Workshop‘ in Google you will get a link to our page which has details on joining us. This is great but if you join DW you also end up automatically joining rtBlogs network. One of our blogs is Facebooknol which is dedicated to Facebook related to posts.

If you enter on Google ‘Join Facebooknol‘ you will not get a link to Facebooknol for joining up. I was unaware of this until I used Walk2Web which did not show ‘how to join’ links with all our blogs except Dw when crawling through their internal links. 😉

Guess it helped us a bit to figure our and start working on a plan to clean up our blogs better. Hope you too find it useful. Do let me know through your comments.

Link: Walk2Web